-- Logs begin at Fri 2022-05-27 18:43:59 UTC, end at Fri 2022-05-27 20:17:26 UTC. -- May 27 19:28:58.912468 dv-cinder-839793-12-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 27 19:28:58.926627 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 27 19:28:58.928665 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri May 27 19:28:58 2022] *** May 27 19:28:58.928665 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 27 19:28:58.928665 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 27 19:28:58.928665 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: nodename: dv-cinder-839793-12-iscsi May 27 19:28:58.928665 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: machine: x86_64 May 27 19:28:58.928665 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: clock source: unix May 27 19:28:58.928665 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: pcre jit disabled May 27 19:28:58.928665 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: detected number of CPU cores: 4 May 27 19:28:58.928665 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: current working directory: / May 27 19:28:58.928665 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: detected binary path: /usr/bin/uwsgi-core May 27 19:28:58.929368 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: your processes number limit is 47680 May 27 19:28:58.929368 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: your memory page size is 4096 bytes May 27 19:28:58.929368 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: detected max file descriptor number: 2048 May 27 19:28:58.929368 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: lock engine: pthread robust mutexes May 27 19:28:58.929368 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: thunder lock: enabled May 27 19:28:58.929368 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 27 19:28:58.929368 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 27 19:28:58.978041 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: Python main interpreter initialized at 0x5640e41b59c0 May 27 19:28:58.978041 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: python threads support enabled May 27 19:28:58.978041 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: your server socket listen backlog is limited to 100 connections May 27 19:28:58.978041 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: your mercy for graceful operations on workers is 90 seconds May 27 19:28:58.978448 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: mapped 403077 bytes (393 KB) for 2 cores May 27 19:28:58.978527 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: *** Operational MODE: preforking *** May 27 19:28:58.978777 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: *** uWSGI is running in multiple interpreter mode *** May 27 19:28:58.978777 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: spawned uWSGI master process (pid: 80622) May 27 19:28:58.979004 dv-cinder-839793-12-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 27 19:28:58.980159 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: spawned uWSGI worker 1 (pid: 80634, cores: 1) May 27 19:28:58.980159 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: spawned uWSGI worker 2 (pid: 80635, cores: 1) May 27 19:28:58.980159 dv-cinder-839793-12-iscsi devstack@keystone.service[80622]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 27 19:29:04.756361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 19:29:04.756361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: from cryptography.utils import int_from_bytes May 27 19:29:04.757215 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 19:29:04.757215 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: from cryptography.utils import int_from_bytes May 27 19:29:04.757342 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 19:29:04.757342 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: from cryptography.utils import int_from_bytes May 27 19:29:04.757342 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 19:29:04.757342 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: from cryptography.utils import int_from_bytes May 27 19:29:04.807454 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 27 19:29:04.807621 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 27 19:29:04.807712 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 27 19:29:04.807802 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 27 19:29:04.807872 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] command line args: [] {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 27 19:29:04.807939 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] command line args: [] {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 27 19:29:04.808000 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 27 19:29:04.808194 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 27 19:29:04.808299 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ================================================================================ {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 27 19:29:04.808381 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ================================================================================ {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 27 19:29:04.808462 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] admin_token = **** {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.808574 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] admin_token = **** {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.808662 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] config_dir = [] {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.808815 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] config_dir = [] {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.808904 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.809013 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] config_source = [] {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.809093 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.809171 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] debug = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.809250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] config_source = [] {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.809361 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] debug = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.809449 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.809696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.809828 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.809933 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.810042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.810153 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.810257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.810364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.810460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.810554 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.810666 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] list_limit = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.810768 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] list_limit = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.810870 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] log_config_append = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.810972 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] log_config_append = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.811075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.811183 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.811290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] log_dir = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.811386 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] log_dir = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.811487 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] log_file = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.811589 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] log_file = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.812231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.812317 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.812505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.812628 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.812843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.812936 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.813084 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.813224 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.813308 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.813437 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.813514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.813640 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.813717 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.813846 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.813929 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.814003 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.814126 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.814203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.814332 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.814409 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.814409 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.814608 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.814689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.814770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.814898 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.814985 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.815058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.815180 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.815263 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.815342 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.815604 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.815690 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.815787 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] publish_errors = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.815876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] publish_errors = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.816003 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.816118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.816230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.816361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.816445 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.816524 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.816639 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.816849 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.816849 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.817037 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.817120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] standard_threads = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.817195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] standard_threads = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.817332 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.817432 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.817591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.817685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.817765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.817842 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.817920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] use_journal = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.817992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] use_journal = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.818069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] use_json = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.818149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] use_json = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.818222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] use_stderr = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.818292 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] use_stderr = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.818389 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] use_syslog = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.818474 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] use_syslog = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.818615 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.818692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 19:29:04.818870 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.818938 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.819104 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.819172 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.819365 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.819447 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.819599 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.819681 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.819840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.819915 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.820243 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.820454 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.820454 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.820600 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.820661 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.820720 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.820855 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.821017 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.821096 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.821171 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.821251 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.821330 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.821471 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.821561 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.821714 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.821796 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.821947 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.822027 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.822188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.822268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.822430 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.822526 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.822662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.822757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.822909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.822996 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.823158 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.823242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.823396 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.823480 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.823627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.823707 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.823875 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.823953 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.824131 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.824209 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.824379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.824466 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.824623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.824718 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.824849 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.824935 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.825101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.825172 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.825328 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.825396 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.825559 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.825626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.825804 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.825873 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.826048 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.826227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.826397 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.826515 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.826642 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.826729 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.826798 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.826870 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.826995 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.827081 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.827156 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.827274 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.828002 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.828165 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.828449 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.828614 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.828702 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.828887 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.829079 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.829209 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.829334 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.829471 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.829597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.829770 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.829964 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.830085 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.830179 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.830276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.830276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.830536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.830616 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.830696 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.830823 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.830907 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.830989 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.831127 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.831217 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.831361 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.831442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.831606 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.831696 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.831835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.831917 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.832070 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.832157 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.832304 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.832393 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.832538 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.832617 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.832769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.832848 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.833027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.833126 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.833256 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.833337 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.833491 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.833570 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.833720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.833807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.833950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.834029 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.834184 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.834345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.834423 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.834548 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.834640 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.834772 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.834851 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.834935 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.835101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.835181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.835335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.835416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.835564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.835642 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.835803 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.835891 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.836040 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.836121 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.836293 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.836372 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.836527 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.836620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.836760 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.836853 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.837013 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.837140 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.837260 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.837341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.837496 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.837584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.837729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.837808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.837970 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.838052 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.838204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.838292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.838436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.838517 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.838669 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.838751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.838902 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.838992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.839135 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.839216 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.839368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.839447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.839604 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.839698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.839842 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.839924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.840072 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.840151 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.840310 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.840400 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.840542 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.840622 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.840773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.840860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.841025 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.841127 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.841255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.841335 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.841481 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.841560 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.841956 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.842053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.842186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.842265 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.842411 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.842491 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.842637 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.842725 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.842864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.842979 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.843149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.843368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.843446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.843532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.843681 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.843768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.843940 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.844020 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.844179 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.844267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.844409 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] auth.external = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.844489 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.844636 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.844721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.844916 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.845023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.845170 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.845249 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] auth.external = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.845399 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] auth.password = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.845490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.845627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] auth.token = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.845715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.845866 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.845954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.846110 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.846199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] auth.password = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.846350 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] auth.token = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.846430 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.846576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.846658 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.846816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.846902 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.847059 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.847146 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.847300 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.847378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.847514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] credential.caching = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.847594 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.847759 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.847838 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.847992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.848082 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.848228 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] credential.caching = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.848309 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.848461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.848541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.848703 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.848788 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.848934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.849048 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.849197 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.849277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.849445 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.849527 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.849677 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.849757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.849926 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.850008 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.850179 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.850260 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.850416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.850498 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.850667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.850749 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.850952 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.851064 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.851247 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.851336 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.851547 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] federation.caching = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.851627 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.851781 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.851865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.852036 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.852117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.852273 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] federation.caching = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.852353 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.852499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.852587 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.852743 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.852823 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.852995 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.853079 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.853234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.853363 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.853472 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.853553 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.853710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.853790 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.853948 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.854030 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.854188 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.854268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.854417 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] identity.caching = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.854497 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.854662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.854745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.854906 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.854987 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.855136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.855216 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] identity.caching = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.855367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.855450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.855608 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.855789 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.855867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.855950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.856056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.856145 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.856300 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.856384 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.856531 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.856623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.856767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.856847 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.857018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.857103 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.857251 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.857349 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.857493 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.857575 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.857734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.857815 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.857965 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.858053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.858216 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.858304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.858460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.858543 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.858785 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.858873 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.859022 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.859185 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.859267 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.859500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.859581 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.859776 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.859858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.860009 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.860088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.860242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.860323 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.860471 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.860559 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.860712 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.860795 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.860945 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.861100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.861251 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.861374 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.861539 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.861673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.861806 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.861932 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.862070 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.862208 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.862336 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.862462 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.862587 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.862713 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.862849 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.862975 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.863103 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.863230 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.863357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.863487 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.863622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.863748 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.863873 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.864009 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.864138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.864266 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.864403 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.864530 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.864655 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.864781 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.864872 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.865064 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.865206 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.865342 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.865471 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.865601 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.865728 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.865865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.866002 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.866136 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.866263 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.866390 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.866518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.866648 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.866782 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.866872 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.866938 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.866999 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.867061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.user = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.867122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.867122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.867243 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.867310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.867372 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.867507 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.867573 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.867573 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.867693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.user = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.867926 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.868054 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.868182 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.868349 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.868474 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.868603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.868743 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.868870 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.869013 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.869153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.869280 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.869405 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.869528 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.869652 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.869777 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.869910 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.870035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.870157 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.870282 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.870416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.870542 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.870742 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.870875 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.870999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.871123 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.871248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.871372 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.871504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.871629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.871760 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.871886 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.872010 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.872141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.872283 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.872409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.872534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.872657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.872781 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.872907 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.873094 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.873224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.873352 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.873434 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.873497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.873558 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.873628 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.873693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.873754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.873903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.874030 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.874158 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.874797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.874927 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.875056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.875183 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.875310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.875445 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.875582 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.875730 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.875815 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.875891 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] resource.caching = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.875964 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.876037 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.876120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.876288 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.876371 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.876522 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.876603 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.876768 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.876857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.877043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.877128 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.877280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.877362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.877519 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.877618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.877751 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.877831 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] resource.caching = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.877986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.878068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.878218 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] role.caching = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.878307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.878457 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.878537 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] role.driver = sql {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.878680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.878760 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.878926 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.879015 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.879164 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.879246 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.879406 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.879486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.879640 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.879728 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.879880 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.879960 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.880112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] role.caching = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.880190 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.880351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.880439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] role.driver = sql {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.880580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.880661 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.880813 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.880893 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.881084 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.881182 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.881329 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.881411 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.881572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.881653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.881814 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.881897 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.882181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.882458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.882553 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.882684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.882773 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.882871 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.882955 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.883297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.883446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.883524 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.883608 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.883686 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.883760 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.883834 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.883908 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.888235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.888370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.888458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.888589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.888669 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.888743 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.888818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.888902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.888995 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.888995 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.889125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.889198 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.889271 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.889271 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.889421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.889421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.889554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.889626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.889699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.889780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.889902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.890465 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.890546 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.890638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] token.caching = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.890779 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.890911 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.891035 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.891148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.891256 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.891364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.891484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.891594 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.891699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.891805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.891910 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.892022 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.892148 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.892258 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.892367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.892474 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.892582 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] token.caching = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.892687 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.892810 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.892918 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.893109 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.893218 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.893326 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.893438 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.893562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.893672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.893779 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.893779 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.893982 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.894104 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.894238 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.894363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.894484 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.894484 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.894712 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.894837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] database.connection = **** {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.894981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.894981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.895213 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.895325 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.895438 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.895559 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.895872 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.895872 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.896224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.896357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.896486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.896610 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.896740 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.896870 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.897046 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.897177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.897298 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] database.connection = **** {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.897435 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.897576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.897703 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.897828 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.897955 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.898080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.898209 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.898338 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.898464 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.898553 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.898728 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.898861 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.898988 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.899091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.899182 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.899311 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.899453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.899570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.899685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.899796 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.899907 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80634) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 27 19:29:04.900024 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.900024 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.900024 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.900024 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.900024 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.900024 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.900024 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.900024 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 19:29:04.900024 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80635) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 27 19:29:04.926315 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:04.926533 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:04.926756 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:04.928633 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:04.928862 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:04.929148 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:04.938297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:04.943656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:04.968183 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:04.968505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:04.973764 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:04.973999 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.023445 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.024003 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.078345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.078621 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.079125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.079362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.147496 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.147722 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.147838 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.147922 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.147999 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.148123 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.148218 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.148336 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.148420 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.148549 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.148632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.148760 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.173244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.173476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.173572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.173699 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.173790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.173916 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.174003 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.174125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.174211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.174333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.174428 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.174541 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.174625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.174752 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.187458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.192105 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.200211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.200443 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.200669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.200890 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.201137 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.201354 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.201569 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80635) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.203311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.203530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.203807 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.204030 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.204244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.204455 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.204664 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80634) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 19:29:05.221696 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.221866 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.221948 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 19:29:05.222023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 19:29:05.222228 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.222311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.222425 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 19:29:05.222516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 19:29:05.222692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.222773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.222948 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.223029 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.223205 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.223311 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.223451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.223533 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.235300 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.235568 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.235834 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.236114 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.237407 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.237698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.237963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.238243 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.244403 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.246414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.247712 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.247992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.248273 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.248550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.248837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.249122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.250085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.250309 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.250550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.250728 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.250864 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.250997 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.251142 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.251284 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.251417 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.251548 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.251722 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.251988 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.252259 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.252526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.252795 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.253085 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.277868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.279550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.279713 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.279911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.281439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.281709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.285417 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.285696 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.285974 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.286963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.287236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.287509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.293340 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.293625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.294832 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.295114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.298826 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.299240 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.299533 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.299815 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.300092 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.300367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.300514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.300654 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.300785 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.300911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.301105 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.301325 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.301598 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.301869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.302133 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.302394 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.320112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.320399 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.321642 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.321923 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.325135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.326552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.329020 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.329258 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.330190 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.330482 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.334449 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.335582 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.338079 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.339207 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.342128 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.342428 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.342720 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.343046 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.343326 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.343556 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.343686 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.343836 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.343963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.344284 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.344731 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.345241 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.364073 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.364370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.364636 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.364892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.365113 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.365386 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.365653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.365909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.373028 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.373965 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.375319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.375599 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.376380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.376657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.378681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.378959 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.379236 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.379519 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.379735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.379867 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.379997 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.380274 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.380550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.380820 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.394051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.394336 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.394535 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.394701 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.394832 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.394959 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.395089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.395216 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.395351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.395480 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.395607 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.395733 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.395860 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.396121 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.414229 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.414806 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.417543 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.418119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.420869 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.422249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.423028 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.423562 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.424674 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.424980 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.425195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.425411 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.425592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.425805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.432781 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.433170 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.434420 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.434786 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.438340 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.438835 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.439229 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.439417 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.439546 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.439896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.440366 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.440833 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.448798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.449112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.449508 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.450868 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.451155 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.451546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.457619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80635) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.457899 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.458177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.458461 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.458735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.459005 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.459291 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.459473 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80634) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 19:29:05.459606 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.459744 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.459883 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.460020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.460150 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.460282 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.460564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.460695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.460819 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.460968 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.461092 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.461214 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.461354 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80635) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.461440 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.461615 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.461837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.462065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.462318 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.462573 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.462840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80634) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 19:29:05.502285 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5640e41b59c0 pid: 80635 (default app) May 27 19:29:05.503204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5640e41b59c0 pid: 80634 (default app) May 27 19:29:05.505753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd031dc9-ebc3-409e-b9e9-4c9ae684cfc0 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:05.506049 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd031dc9-ebc3-409e-b9e9-4c9ae684cfc0 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:05.506330 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd031dc9-ebc3-409e-b9e9-4c9ae684cfc0 None None] PATH_INFO: `/v3` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:05.510737 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Fri May 27 19:29:05 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:14.687783 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5422e3-df56-4741-b63a-68dbd6c49e27 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:14.688315 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5422e3-df56-4741-b63a-68dbd6c49e27 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:14.688315 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5422e3-df56-4741-b63a-68dbd6c49e27 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:14.692720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:14.698405 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa16d5-b224-4a56-8bf4-9462d355eea3 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:14.698875 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa16d5-b224-4a56-8bf4-9462d355eea3 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:14.698961 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa16d5-b224-4a56-8bf4-9462d355eea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:14.769897 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: INFO dbcounter [None req-51fa16d5-b224-4a56-8bf4-9462d355eea3 None None] Registered counter for database keystone May 27 19:29:14.791147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG oslo_db.sqlalchemy.engines [None req-51fa16d5-b224-4a56-8bf4-9462d355eea3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80635) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 27 19:29:14.797810 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writer thread running {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 27 19:29:14.979483 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG passlib.handlers.bcrypt [None req-51fa16d5-b224-4a56-8bf4-9462d355eea3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80635) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 27 19:29:14.979906 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG passlib.handlers.bcrypt [None req-51fa16d5-b224-4a56-8bf4-9462d355eea3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80635) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 27 19:29:15.037655 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-51fa16d5-b224-4a56-8bf4-9462d355eea3 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:15.039176 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-51fa16d5-b224-4a56-8bf4-9462d355eea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:15.176277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:14 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 27 19:29:15.353887 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5422e3-df56-4741-b63a-68dbd6c49e27 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:15.424249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: INFO dbcounter [None req-eb5422e3-df56-4741-b63a-68dbd6c49e27 None None] Registered counter for database keystone May 27 19:29:15.444710 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG oslo_db.sqlalchemy.engines [None req-eb5422e3-df56-4741-b63a-68dbd6c49e27 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80634) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 27 19:29:15.451086 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writer thread running {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 27 19:29:15.574004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5422e3-df56-4741-b63a-68dbd6c49e27 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:15.578993 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f2ad4-378f-4deb-bc45-ba851ec8c425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:15.579662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f2ad4-378f-4deb-bc45-ba851ec8c425 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:15.579946 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f2ad4-378f-4deb-bc45-ba851ec8c425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:15.580224 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f2ad4-378f-4deb-bc45-ba851ec8c425 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:15.588879 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f2ad4-378f-4deb-bc45-ba851ec8c425 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:15.641237 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1b1f2ad4-378f-4deb-bc45-ba851ec8c425 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 27 19:29:15.641499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f2ad4-378f-4deb-bc45-ba851ec8c425 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:15.652276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Fri May 27 19:29:15 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 19:29:15.658667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa16d5-b224-4a56-8bf4-9462d355eea3 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:15.673666 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa16d5-b224-4a56-8bf4-9462d355eea3 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:15.678101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c547e-c6b9-462b-b438-553882bed2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:15.679226 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c547e-c6b9-462b-b438-553882bed2ba admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:15.679514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c547e-c6b9-462b-b438-553882bed2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:15.679793 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c547e-c6b9-462b-b438-553882bed2ba admin admin] PATH_INFO: `/v3/projects/6c2fc958121b4494a2ef337a1cb0b9b4` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:15.681348 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c547e-c6b9-462b-b438-553882bed2ba admin admin] RBAC: Authorizing `identity:get_project(project_id=6c2fc958121b4494a2ef337a1cb0b9b4)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:15.841440 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: WARNING keystone.common.rbac_enforcer.enforcer [None req-852c547e-c6b9-462b-b438-553882bed2ba admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 27 19:29:15.842078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c547e-c6b9-462b-b438-553882bed2ba admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:15.846892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Fri May 27 19:29:15 2022] GET /identity/v3/projects/6c2fc958121b4494a2ef337a1cb0b9b4 => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:18.062073 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea76e91-3853-43f6-a56b-e30ca378419d None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:18.062611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea76e91-3853-43f6-a56b-e30ca378419d None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:18.062993 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea76e91-3853-43f6-a56b-e30ca378419d None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:18.067264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 19:29:18.073906 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d9872-de03-4ef7-8959-e5c22fd7706b None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:18.074558 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d9872-de03-4ef7-8959-e5c22fd7706b None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:18.074558 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d9872-de03-4ef7-8959-e5c22fd7706b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:18.094655 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-374d9872-de03-4ef7-8959-e5c22fd7706b None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:18.096288 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-374d9872-de03-4ef7-8959-e5c22fd7706b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:18.104350 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:18 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 27 19:29:18.129543 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea76e91-3853-43f6-a56b-e30ca378419d None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:18.141967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea76e91-3853-43f6-a56b-e30ca378419d None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:18.146323 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2bed8e-2710-49f4-9ed3-e5ac096eb006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:18.146967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bed8e-2710-49f4-9ed3-e5ac096eb006 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:18.147250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bed8e-2710-49f4-9ed3-e5ac096eb006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:18.147527 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2bed8e-2710-49f4-9ed3-e5ac096eb006 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:18.155631 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2bed8e-2710-49f4-9ed3-e5ac096eb006 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:18.157465 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2bed8e-2710-49f4-9ed3-e5ac096eb006 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:18.166386 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Fri May 27 19:29:18 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 19:29:18.173315 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d9872-de03-4ef7-8959-e5c22fd7706b None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:18.182379 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d9872-de03-4ef7-8959-e5c22fd7706b None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:18.187887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4527f97-d488-4754-9c4a-c97c91545d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:18.188780 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4527f97-d488-4754-9c4a-c97c91545d5d admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:18.189097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4527f97-d488-4754-9c4a-c97c91545d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:18.189375 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4527f97-d488-4754-9c4a-c97c91545d5d admin admin] PATH_INFO: `/v3/users/736ecd08572d41d59ba0fe2db80e313a` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:18.190774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4527f97-d488-4754-9c4a-c97c91545d5d admin admin] RBAC: Authorizing `identity:get_user(user_id=736ecd08572d41d59ba0fe2db80e313a)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:18.192627 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4527f97-d488-4754-9c4a-c97c91545d5d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:18.194255 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Fri May 27 19:29:18 2022] GET /identity/v3/users/736ecd08572d41d59ba0fe2db80e313a => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:29:20.488591 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24898203-2baf-45d0-882e-02e5fbb0c535 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:20.489135 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24898203-2baf-45d0-882e-02e5fbb0c535 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:20.489219 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24898203-2baf-45d0-882e-02e5fbb0c535 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:20.490543 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:20.495781 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768ca2b-cce3-4a97-b952-c5686604d95e None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:20.496207 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768ca2b-cce3-4a97-b952-c5686604d95e None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:20.496350 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8768ca2b-cce3-4a97-b952-c5686604d95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:20.513290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-8768ca2b-cce3-4a97-b952-c5686604d95e None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:20.514461 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-8768ca2b-cce3-4a97-b952-c5686604d95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:20.594962 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b09ffb-99bd-43ac-8fe5-b95ca42ed172 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:20.595273 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b09ffb-99bd-43ac-8fe5-b95ca42ed172 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:20.595557 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b09ffb-99bd-43ac-8fe5-b95ca42ed172 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:20.597020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 19:29:20.598842 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 6/12] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:20.602116 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7049b2c-65c5-4e2a-8a54-42dbf6d1238c None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:20.602408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7049b2c-65c5-4e2a-8a54-42dbf6d1238c None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:20.602693 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7049b2c-65c5-4e2a-8a54-42dbf6d1238c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:20.621712 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768ca2b-cce3-4a97-b952-c5686604d95e None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:20.643722 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG passlib.handlers.bcrypt [None req-a7049b2c-65c5-4e2a-8a54-42dbf6d1238c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80634) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 27 19:29:20.644153 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG passlib.handlers.bcrypt [None req-a7049b2c-65c5-4e2a-8a54-42dbf6d1238c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80634) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 27 19:29:20.647148 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:20.691335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-a7049b2c-65c5-4e2a-8a54-42dbf6d1238c None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:20.692938 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-a7049b2c-65c5-4e2a-8a54-42dbf6d1238c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:20.727846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8768ca2b-cce3-4a97-b952-c5686604d95e None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:20.762247 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445da931-2833-4cef-857b-6ec46fb83be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:20.762966 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445da931-2833-4cef-857b-6ec46fb83be6 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:20.763252 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445da931-2833-4cef-857b-6ec46fb83be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:20.763535 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445da931-2833-4cef-857b-6ec46fb83be6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:20.775458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445da931-2833-4cef-857b-6ec46fb83be6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:29:20.786770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-445da931-2833-4cef-857b-6ec46fb83be6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:29:20.786770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:20.786770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:20.786770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:20.786770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:20.786770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:20.786770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:20.786770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:20.786770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:20.786770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:20.786770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:20.786770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:20.786770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:29:20.786770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:29:20.786770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:29:20.786770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:29:20.787750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:29:20.787750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:29:20.787750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:29:20.787750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:29:20.787750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:20.787750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:20.787750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:20.787750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:20.787750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:20.787750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:20.787750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:20.787750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:29:20.787750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:20.787750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:20.787750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:20.787750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:20.788730 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:20.788730 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:20.788730 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:20.788730 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:20.788730 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:29:20.788730 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:29:20.788730 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:29:20.788730 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:29:20.788730 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:29:20.788730 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:29:20.788730 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:29:20.788730 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:29:20.788730 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 7/13] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:29:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:29:20.793804 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445da931-2833-4cef-857b-6ec46fb83be6 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:20.794106 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 7/14] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:20.816127 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7049b2c-65c5-4e2a-8a54-42dbf6d1238c None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:20.880612 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:20.900277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445da931-2833-4cef-857b-6ec46fb83be6 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:20.923043 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7049b2c-65c5-4e2a-8a54-42dbf6d1238c None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:20.925520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:20.937787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d2dde-ef99-4539-b634-4a15f0496843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:20.938521 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d2dde-ef99-4539-b634-4a15f0496843 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:20.938781 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d2dde-ef99-4539-b634-4a15f0496843 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:20.939057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d2dde-ef99-4539-b634-4a15f0496843 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:20.939882 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d2dde-ef99-4539-b634-4a15f0496843 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:20.940190 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d2dde-ef99-4539-b634-4a15f0496843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:20.941681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4d2dde-ef99-4539-b634-4a15f0496843 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:20.954964 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 8/15] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:29:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:29:20.960424 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d2dde-ef99-4539-b634-4a15f0496843 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:20.963690 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1481b7e-99ad-4adb-954c-ab3ad485a19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:20.964667 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1481b7e-99ad-4adb-954c-ab3ad485a19a None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:20.964972 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1481b7e-99ad-4adb-954c-ab3ad485a19a None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:20.965391 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1481b7e-99ad-4adb-954c-ab3ad485a19a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:21.035265 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1481b7e-99ad-4adb-954c-ab3ad485a19a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:21.036732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1481b7e-99ad-4adb-954c-ab3ad485a19a None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:21.063856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d2dde-ef99-4539-b634-4a15f0496843 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:21.098773 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc9300-bf71-4bd1-9155-a39b8a0cc44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:21.099751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc9300-bf71-4bd1-9155-a39b8a0cc44c None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:21.100043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc9300-bf71-4bd1-9155-a39b8a0cc44c None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:21.100324 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc9300-bf71-4bd1-9155-a39b8a0cc44c None admin] PATH_INFO: `/v3/users/736ecd08572d41d59ba0fe2db80e313a` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:21.101602 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc9300-bf71-4bd1-9155-a39b8a0cc44c None admin] RBAC: Authorizing `identity:get_user(user_id=736ecd08572d41d59ba0fe2db80e313a)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:21.103096 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc9300-bf71-4bd1-9155-a39b8a0cc44c None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:21.104865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 9/16] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:29:20 2022] GET /identity/v3/users/736ecd08572d41d59ba0fe2db80e313a => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:29:21.110868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc9300-bf71-4bd1-9155-a39b8a0cc44c None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:21.127175 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:21.140250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 8/17] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:29:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 19:29:21.145949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1481b7e-99ad-4adb-954c-ab3ad485a19a None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:21.148854 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:21.214484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc9300-bf71-4bd1-9155-a39b8a0cc44c None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:21.247218 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1481b7e-99ad-4adb-954c-ab3ad485a19a None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:21.252534 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ff260-12b4-4125-a1e1-443c6d04bce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:21.253391 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ff260-12b4-4125-a1e1-443c6d04bce7 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:21.253682 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ff260-12b4-4125-a1e1-443c6d04bce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:21.253991 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ff260-12b4-4125-a1e1-443c6d04bce7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:21.255198 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ff260-12b4-4125-a1e1-443c6d04bce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:21.256677 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ff260-12b4-4125-a1e1-443c6d04bce7 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:21.258310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 10/18] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:29:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:29:21.264244 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ff260-12b4-4125-a1e1-443c6d04bce7 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:21.283134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90395ac-9334-486c-9380-df2e4b7f246f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:21.283932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90395ac-9334-486c-9380-df2e4b7f246f None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:21.284213 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90395ac-9334-486c-9380-df2e4b7f246f None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:21.284487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90395ac-9334-486c-9380-df2e4b7f246f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:21.303956 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90395ac-9334-486c-9380-df2e4b7f246f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:21.305442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90395ac-9334-486c-9380-df2e4b7f246f None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:21.326780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-a90395ac-9334-486c-9380-df2e4b7f246f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:29:21.326780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:21.326780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:21.326780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:21.326780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:21.326780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:21.326780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:21.326780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:21.326780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:21.326780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:21.326780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:21.326780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:21.326780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:29:21.326780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:29:21.326780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:29:21.327735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:29:21.327735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:21.327735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:21.327735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:21.327735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:21.327735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:21.327735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:21.327735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:21.327735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:29:21.327735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:21.327735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:21.327735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:21.327735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:21.327735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:21.327735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:21.327735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:21.327735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:21.328699 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:29:21.328699 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:29:21.328699 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:29:21.328699 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:29:21.328699 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 9/19] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:29:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:29:21.333907 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90395ac-9334-486c-9380-df2e4b7f246f None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:21.352106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:21.371497 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ff260-12b4-4125-a1e1-443c6d04bce7 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:21.392054 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:21.405218 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b382e00-3d32-4c2c-a19b-cef676c56937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:21.406575 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b382e00-3d32-4c2c-a19b-cef676c56937 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:21.406854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b382e00-3d32-4c2c-a19b-cef676c56937 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:21.407136 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b382e00-3d32-4c2c-a19b-cef676c56937 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:21.408004 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b382e00-3d32-4c2c-a19b-cef676c56937 None admin] RBAC: Adding query filter params (role.id=6505bbb129404436966ee59aa7a01d8e, scope.domain.id=default, user.id=736ecd08572d41d59ba0fe2db80e313a) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:21.408359 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b382e00-3d32-4c2c-a19b-cef676c56937 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:21.409841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b382e00-3d32-4c2c-a19b-cef676c56937 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:21.416987 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 11/20] 10.222.0.13 () {56 vars in 1322 bytes} [Fri May 27 19:29:21 2022] GET /identity/v3/role_assignments?user.id=736ecd08572d41d59ba0fe2db80e313a&scope.domain.id=default&role.id=6505bbb129404436966ee59aa7a01d8e => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:29:21.438351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90395ac-9334-486c-9380-df2e4b7f246f None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:21.475683 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b63a5-91ca-4d3e-9d13-1ecb2ecbbd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:21.476372 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b63a5-91ca-4d3e-9d13-1ecb2ecbbd6d None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:21.476659 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b63a5-91ca-4d3e-9d13-1ecb2ecbbd6d None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:21.476939 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b63a5-91ca-4d3e-9d13-1ecb2ecbbd6d None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:21.477801 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b63a5-91ca-4d3e-9d13-1ecb2ecbbd6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:21.478115 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b63a5-91ca-4d3e-9d13-1ecb2ecbbd6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:21.479568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b63a5-91ca-4d3e-9d13-1ecb2ecbbd6d None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:21.500265 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 10/21] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:29:24.199642 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c24436-89a9-4d52-9985-002ac6e2ae33 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:24.200432 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c24436-89a9-4d52-9985-002ac6e2ae33 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:24.200432 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c24436-89a9-4d52-9985-002ac6e2ae33 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:24.201627 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 12/22] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:24.211249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb0fa0-0f8e-4958-b7ab-f8aaac013db9 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:24.212023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb0fa0-0f8e-4958-b7ab-f8aaac013db9 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:24.212318 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb0fa0-0f8e-4958-b7ab-f8aaac013db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:24.267471 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-dbcb0fa0-0f8e-4958-b7ab-f8aaac013db9 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:24.269517 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-dbcb0fa0-0f8e-4958-b7ab-f8aaac013db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:24.478747 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f54879-4268-4fe9-b207-34753e14c189 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:24.478971 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f54879-4268-4fe9-b207-34753e14c189 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:24.479352 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f54879-4268-4fe9-b207-34753e14c189 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:24.480649 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 13/23] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 19:29:24.489543 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afbee7-9dd4-43aa-838f-2b0c581922ec None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:24.489949 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afbee7-9dd4-43aa-838f-2b0c581922ec None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:24.490186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afbee7-9dd4-43aa-838f-2b0c581922ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:24.532110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-56afbee7-9dd4-43aa-838f-2b0c581922ec None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:24.532110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-56afbee7-9dd4-43aa-838f-2b0c581922ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:24.588602 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 11/24] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 19:29:24.599106 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe685b5a-e870-440d-9d80-d30d5ae6d600 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:24.599379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe685b5a-e870-440d-9d80-d30d5ae6d600 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:24.599673 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe685b5a-e870-440d-9d80-d30d5ae6d600 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:24.601861 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 12/25] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:24 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:24.610365 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4400fc-1f9c-42cf-9f2b-68b1bfafc7ec None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:24.610716 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4400fc-1f9c-42cf-9f2b-68b1bfafc7ec None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:24.610936 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4400fc-1f9c-42cf-9f2b-68b1bfafc7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:24.643825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-0c4400fc-1f9c-42cf-9f2b-68b1bfafc7ec None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:24.645066 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-0c4400fc-1f9c-42cf-9f2b-68b1bfafc7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:24.674353 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 14/26] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 19:29:24.675316 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afbee7-9dd4-43aa-838f-2b0c581922ec None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:24.737765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:24.782728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:24.792100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 13/27] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 19:29:24.793062 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4400fc-1f9c-42cf-9f2b-68b1bfafc7ec None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:24.829372 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afbee7-9dd4-43aa-838f-2b0c581922ec None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:24.890390 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12c256-2877-45f0-9cb7-dd5babcfd2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:24.891824 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12c256-2877-45f0-9cb7-dd5babcfd2ea None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:24.892582 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12c256-2877-45f0-9cb7-dd5babcfd2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:24.893463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12c256-2877-45f0-9cb7-dd5babcfd2ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:24.907049 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e12c256-2877-45f0-9cb7-dd5babcfd2ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:24.909846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e12c256-2877-45f0-9cb7-dd5babcfd2ea None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:24.935316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4400fc-1f9c-42cf-9f2b-68b1bfafc7ec None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:24.936596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-1e12c256-2877-45f0-9cb7-dd5babcfd2ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:29:24.936596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:24.936596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:24.936596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:24.936596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:24.936596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:24.936596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:24.936596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:24.936596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:24.936596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:24.936596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:24.936596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:24.936596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:29:24.936596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:29:24.936596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:29:24.938280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:29:24.938280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:24.938280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:24.938280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:24.938280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:24.938280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:24.938280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:24.938280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:24.938280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:29:24.938280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:24.938280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:24.938280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:24.938280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:24.938280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:24.938280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:24.938280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:24.938280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:24.939256 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:29:24.939256 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:29:24.939256 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:29:24.939256 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:29:24.940128 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 15/28] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:29:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:29:24.942506 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd85cc-79d4-4e16-bf99-3ecae4a87ba8 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:24.942869 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd85cc-79d4-4e16-bf99-3ecae4a87ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:24.943216 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cd85cc-79d4-4e16-bf99-3ecae4a87ba8 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:24.944564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 16/29] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:24.950484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd85cc-79d4-4e16-bf99-3ecae4a87ba8 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:24.988573 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec017bf-6141-4f92-82fd-a1109ae6bba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:24.989706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec017bf-6141-4f92-82fd-a1109ae6bba1 None admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:24.990435 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec017bf-6141-4f92-82fd-a1109ae6bba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:24.991141 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec017bf-6141-4f92-82fd-a1109ae6bba1 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:24.992400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec017bf-6141-4f92-82fd-a1109ae6bba1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:24.994004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec017bf-6141-4f92-82fd-a1109ae6bba1 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:25.068873 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cd85cc-79d4-4e16-bf99-3ecae4a87ba8 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:25.097693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:25.108061 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73254e93-34aa-4131-a7ef-6a0689ec5060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:25.108937 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73254e93-34aa-4131-a7ef-6a0689ec5060 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:25.109248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73254e93-34aa-4131-a7ef-6a0689ec5060 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:25.109521 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73254e93-34aa-4131-a7ef-6a0689ec5060 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:25.120143 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73254e93-34aa-4131-a7ef-6a0689ec5060 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:29:25.129267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-73254e93-34aa-4131-a7ef-6a0689ec5060 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:29:25.129267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:25.129267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:25.129267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:25.129267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:25.129267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:25.129267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:25.129267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:25.129267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:25.129267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:25.129267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:25.129267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:25.129267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:29:25.129267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:29:25.129267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:29:25.129267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:29:25.130167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:29:25.130167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:29:25.130167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:29:25.130167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:29:25.130167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:25.130167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:25.130167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:25.130167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:25.130167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:25.130167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:25.130167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:25.130167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:29:25.130167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:25.130167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:25.130167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:25.130167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:25.130925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:25.130925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:25.130925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:25.130925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:25.130925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:29:25.130925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:29:25.130925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:29:25.130925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:29:25.130925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:29:25.130925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:29:25.130925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:29:25.130925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:29:25.130925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 17/30] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:29:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:29:25.131570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73254e93-34aa-4131-a7ef-6a0689ec5060 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:25.250817 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73254e93-34aa-4131-a7ef-6a0689ec5060 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:25.257973 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 14/31] 10.222.0.13 () {60 vars in 1131 bytes} [Fri May 27 19:29:24 2022] POST /identity/v3/roles => generated 219 bytes in 459 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:29:25.257973 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b631e-7f9f-493f-9934-88eb07a55ace None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:25.257973 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b631e-7f9f-493f-9934-88eb07a55ace None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:25.257973 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b631e-7f9f-493f-9934-88eb07a55ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:25.296194 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-a45b631e-7f9f-493f-9934-88eb07a55ace None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:25.298080 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-a45b631e-7f9f-493f-9934-88eb07a55ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:25.337013 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1178cf-ba19-4c77-abbf-f6ec96036846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:25.337804 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1178cf-ba19-4c77-abbf-f6ec96036846 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:25.338089 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1178cf-ba19-4c77-abbf-f6ec96036846 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:25.338370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1178cf-ba19-4c77-abbf-f6ec96036846 None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:25.339115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1178cf-ba19-4c77-abbf-f6ec96036846 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:25.339418 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1178cf-ba19-4c77-abbf-f6ec96036846 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:25.340847 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1178cf-ba19-4c77-abbf-f6ec96036846 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:25.369227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:25.382822 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 18/32] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:25.384166 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1178cf-ba19-4c77-abbf-f6ec96036846 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:25.441341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:25.468761 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 15/33] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:25.469721 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b631e-7f9f-493f-9934-88eb07a55ace None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:25.507751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1178cf-ba19-4c77-abbf-f6ec96036846 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:25.547271 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8553482-2f24-4e75-afe2-6534552971fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:25.549264 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8553482-2f24-4e75-afe2-6534552971fa None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:25.549827 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8553482-2f24-4e75-afe2-6534552971fa None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:25.550300 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8553482-2f24-4e75-afe2-6534552971fa None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:25.551383 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8553482-2f24-4e75-afe2-6534552971fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:25.551882 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8553482-2f24-4e75-afe2-6534552971fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:25.553613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8553482-2f24-4e75-afe2-6534552971fa None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:25.565082 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 19/34] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:29:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:25.566413 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8553482-2f24-4e75-afe2-6534552971fa None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:25.613292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b631e-7f9f-493f-9934-88eb07a55ace None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:25.660187 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ef1b0-58a3-48ed-9c5f-4745eda2d7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:25.661661 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ef1b0-58a3-48ed-9c5f-4745eda2d7ee None admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:25.661661 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ef1b0-58a3-48ed-9c5f-4745eda2d7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:25.661832 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ef1b0-58a3-48ed-9c5f-4745eda2d7ee None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:25.662873 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ef1b0-58a3-48ed-9c5f-4745eda2d7ee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:25.664490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ef1b0-58a3-48ed-9c5f-4745eda2d7ee None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:25.688146 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8553482-2f24-4e75-afe2-6534552971fa None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:25.700887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:25.732376 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2aeb7-a0b8-4095-81a0-0ac4a64a2065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:25.733936 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2aeb7-a0b8-4095-81a0-0ac4a64a2065 None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:25.734470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2aeb7-a0b8-4095-81a0-0ac4a64a2065 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:25.735096 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2aeb7-a0b8-4095-81a0-0ac4a64a2065 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:25.736034 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2aeb7-a0b8-4095-81a0-0ac4a64a2065 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:25.737750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2aeb7-a0b8-4095-81a0-0ac4a64a2065 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:25.809123 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 16/35] 10.222.0.13 () {60 vars in 1141 bytes} [Fri May 27 19:29:25 2022] POST /identity/v3/projects => generated 301 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:29:25.809123 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ef1b0-58a3-48ed-9c5f-4745eda2d7ee None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:25.834601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:25.935674 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ef1b0-58a3-48ed-9c5f-4745eda2d7ee None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:25.978703 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1db0e-112d-46a4-b5a3-e33db702459c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:25.979615 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1db0e-112d-46a4-b5a3-e33db702459c None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:25.979904 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1db0e-112d-46a4-b5a3-e33db702459c None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:25.980184 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1db0e-112d-46a4-b5a3-e33db702459c None admin] PATH_INFO: `/v3/users/736ecd08572d41d59ba0fe2db80e313a` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:25.981718 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab1db0e-112d-46a4-b5a3-e33db702459c None admin] RBAC: Authorizing `identity:get_user(user_id=736ecd08572d41d59ba0fe2db80e313a)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:25.983453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab1db0e-112d-46a4-b5a3-e33db702459c None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:25.985405 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 17/36] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:29:25 2022] GET /identity/v3/users/736ecd08572d41d59ba0fe2db80e313a => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:25.989256 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 20/37] 10.222.0.13 () {60 vars in 1131 bytes} [Fri May 27 19:29:25 2022] POST /identity/v3/roles => generated 225 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:29:25.989406 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab6701-a3b9-4af5-a292-35b0df562c98 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:25.989406 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab6701-a3b9-4af5-a292-35b0df562c98 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:25.989406 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab6701-a3b9-4af5-a292-35b0df562c98 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:25.989601 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e2f41-3367-447d-ac2e-9bfd5d38d47b None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:25.989601 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e2f41-3367-447d-ac2e-9bfd5d38d47b None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:25.989601 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59e2f41-3367-447d-ac2e-9bfd5d38d47b None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:25.989778 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 18/38] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:25.989840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 21/39] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:25.992421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab6701-a3b9-4af5-a292-35b0df562c98 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:25.998853 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4a4b6-9928-46f1-854f-463667ebe24a None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:25.999780 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4a4b6-9928-46f1-854f-463667ebe24a None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:26.000226 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4a4b6-9928-46f1-854f-463667ebe24a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:26.023776 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-a7e4a4b6-9928-46f1-854f-463667ebe24a None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:26.025782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-a7e4a4b6-9928-46f1-854f-463667ebe24a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:26.130593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:26.141324 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 22/40] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 19:29:26.142758 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b0bf2-c2ed-43d3-801c-d355d4d28e97 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:26.143057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b0bf2-c2ed-43d3-801c-d355d4d28e97 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:26.143345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4b0bf2-c2ed-43d3-801c-d355d4d28e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:26.151426 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab6701-a3b9-4af5-a292-35b0df562c98 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:26.161982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-3d4b0bf2-c2ed-43d3-801c-d355d4d28e97 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:26.163630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-3d4b0bf2-c2ed-43d3-801c-d355d4d28e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:26.197588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76cd03d-fa85-412c-b5f0-97be8e78211c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:26.198402 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cd03d-fa85-412c-b5f0-97be8e78211c None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:26.198695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cd03d-fa85-412c-b5f0-97be8e78211c None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:26.198971 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cd03d-fa85-412c-b5f0-97be8e78211c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:26.200233 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76cd03d-fa85-412c-b5f0-97be8e78211c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:26.201919 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76cd03d-fa85-412c-b5f0-97be8e78211c None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:26.203475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 19/41] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:29:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:29:26.204989 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76cd03d-fa85-412c-b5f0-97be8e78211c None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:26.252190 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 23/42] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:26.252852 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b0bf2-c2ed-43d3-801c-d355d4d28e97 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:26.281578 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:26.320093 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76cd03d-fa85-412c-b5f0-97be8e78211c None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:26.353666 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953dbc5-b51b-45af-b91e-caf2fa68863b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:26.354461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953dbc5-b51b-45af-b91e-caf2fa68863b None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:26.354728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4b0bf2-c2ed-43d3-801c-d355d4d28e97 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:26.354823 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953dbc5-b51b-45af-b91e-caf2fa68863b None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:26.355015 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953dbc5-b51b-45af-b91e-caf2fa68863b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:26.356168 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953dbc5-b51b-45af-b91e-caf2fa68863b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:26.357703 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953dbc5-b51b-45af-b91e-caf2fa68863b None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:26.359830 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 20/43] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:29:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:26.360575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953dbc5-b51b-45af-b91e-caf2fa68863b None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:26.386974 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:26.391461 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a137c-0e9c-4532-92ce-ba51e565655c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:26.398762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a137c-0e9c-4532-92ce-ba51e565655c None admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:26.398762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a137c-0e9c-4532-92ce-ba51e565655c None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:26.398762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a137c-0e9c-4532-92ce-ba51e565655c None admin] PATH_INFO: `/v3/domains/default/users/736ecd08572d41d59ba0fe2db80e313a/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:26.398762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a137c-0e9c-4532-92ce-ba51e565655c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=736ecd08572d41d59ba0fe2db80e313a, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:26.398762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a137c-0e9c-4532-92ce-ba51e565655c None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:26.462268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953dbc5-b51b-45af-b91e-caf2fa68863b None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:26.473542 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 24/44] 10.222.0.13 () {58 vars in 1361 bytes} [Fri May 27 19:29:26 2022] PUT /identity/v3/domains/default/users/736ecd08572d41d59ba0fe2db80e313a/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:29:26.474374 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a137c-0e9c-4532-92ce-ba51e565655c None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:26.507909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb646e-d992-49a6-bb03-2cbed0c4ec49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:26.508612 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb646e-d992-49a6-bb03-2cbed0c4ec49 None admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:26.508899 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb646e-d992-49a6-bb03-2cbed0c4ec49 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:26.509229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb646e-d992-49a6-bb03-2cbed0c4ec49 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:26.510150 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eb646e-d992-49a6-bb03-2cbed0c4ec49 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:26.511575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eb646e-d992-49a6-bb03-2cbed0c4ec49 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:26.586880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:26.589994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 21/45] 10.222.0.13 () {60 vars in 1131 bytes} [Fri May 27 19:29:26 2022] POST /identity/v3/roles => generated 223 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:29:26.601162 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a137c-0e9c-4532-92ce-ba51e565655c None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:26.643839 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b3341-7c6e-4d36-8462-cba6f546a25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:26.644460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b3341-7c6e-4d36-8462-cba6f546a25e None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:26.644755 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b3341-7c6e-4d36-8462-cba6f546a25e None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:26.645086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b3341-7c6e-4d36-8462-cba6f546a25e None admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:26.645831 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b3341-7c6e-4d36-8462-cba6f546a25e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:26.647435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b3341-7c6e-4d36-8462-cba6f546a25e None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:26.765254 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 25/46] 10.222.0.13 () {60 vars in 1141 bytes} [Fri May 27 19:29:26 2022] POST /identity/v3/projects => generated 312 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:29:28.761196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c659a-ea1c-46f9-8070-01b42e09b65c None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:28.761766 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c659a-ea1c-46f9-8070-01b42e09b65c None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:28.761766 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c659a-ea1c-46f9-8070-01b42e09b65c None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:28.763122 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 22/47] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:28.769134 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8ba2d-b55b-45fc-8209-48296ca49419 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:28.769560 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8ba2d-b55b-45fc-8209-48296ca49419 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:28.769712 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb8ba2d-b55b-45fc-8209-48296ca49419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:28.789717 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-fbb8ba2d-b55b-45fc-8209-48296ca49419 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:28.791292 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-fbb8ba2d-b55b-45fc-8209-48296ca49419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:28.868313 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 26/48] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:28.893091 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c659a-ea1c-46f9-8070-01b42e09b65c None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:28.979017 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:29.003216 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c659a-ea1c-46f9-8070-01b42e09b65c None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:29.045776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1647c-43d2-4116-b040-0956756b4b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:29.047015 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1647c-43d2-4116-b040-0956756b4b7e None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:29.047409 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1647c-43d2-4116-b040-0956756b4b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:29.049413 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1647c-43d2-4116-b040-0956756b4b7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:29.066906 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d1647c-43d2-4116-b040-0956756b4b7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:29:29.079709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-11d1647c-43d2-4116-b040-0956756b4b7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:29:29.079709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:29.079709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:29.079709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:29.079709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:29.079709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:29.079709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:29.079709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:29.079709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:29.079709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:29.079709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:29.079709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:29.079709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:29:29.079709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:29:29.079709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:29:29.079709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:29:29.080676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:29:29.080676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:29:29.080676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:29:29.080676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:29:29.080676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:29.080676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:29.080676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:29.080676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:29.080676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:29.080676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:29.080676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:29.080676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:29:29.080676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:29.080676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:29.080676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:29.080676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:29.081535 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:29.081535 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:29.081535 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:29.081535 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:29.081535 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:29:29.081535 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:29:29.081535 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:29:29.081535 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:29:29.081535 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:29:29.081535 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:29:29.081535 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:29:29.081535 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:29:29.082523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 23/49] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:29:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:29:29.088436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8ba2d-b55b-45fc-8209-48296ca49419 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:29.185568 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:29.206346 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ab199-617b-41d0-8d17-a6ee25d604b2 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:29.206632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ab199-617b-41d0-8d17-a6ee25d604b2 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:29.206922 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ab199-617b-41d0-8d17-a6ee25d604b2 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:29.208435 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 24/50] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 19:29:29.213469 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df4d1c-3b34-4b55-b84b-d089e6312d9e None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:29.213772 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df4d1c-3b34-4b55-b84b-d089e6312d9e None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:29.214050 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df4d1c-3b34-4b55-b84b-d089e6312d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:29.219890 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb8ba2d-b55b-45fc-8209-48296ca49419 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:29.232268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-31df4d1c-3b34-4b55-b84b-d089e6312d9e None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:29.233552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-31df4d1c-3b34-4b55-b84b-d089e6312d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:29.255310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2992f93-1f8b-42bb-85eb-4ee608bf6d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:29.255983 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2992f93-1f8b-42bb-85eb-4ee608bf6d89 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:29.256260 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2992f93-1f8b-42bb-85eb-4ee608bf6d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:29.256545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2992f93-1f8b-42bb-85eb-4ee608bf6d89 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:29.257331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2992f93-1f8b-42bb-85eb-4ee608bf6d89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:29.257640 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2992f93-1f8b-42bb-85eb-4ee608bf6d89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:29.259026 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2992f93-1f8b-42bb-85eb-4ee608bf6d89 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:29.268123 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 27/51] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:29:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:29.273634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2992f93-1f8b-42bb-85eb-4ee608bf6d89 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:29.309764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 25/52] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 19:29:29.337378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df4d1c-3b34-4b55-b84b-d089e6312d9e None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:29.380880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2992f93-1f8b-42bb-85eb-4ee608bf6d89 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:29.414779 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:29.417977 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:29.420522 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56554d2-c1fb-4790-93dd-aa8d913dad60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:29.421646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56554d2-c1fb-4790-93dd-aa8d913dad60 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:29.421918 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56554d2-c1fb-4790-93dd-aa8d913dad60 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:29.422196 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56554d2-c1fb-4790-93dd-aa8d913dad60 None admin] PATH_INFO: `/v3/users/736ecd08572d41d59ba0fe2db80e313a` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:29.423415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56554d2-c1fb-4790-93dd-aa8d913dad60 None admin] RBAC: Authorizing `identity:get_user(user_id=736ecd08572d41d59ba0fe2db80e313a)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:29.424882 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56554d2-c1fb-4790-93dd-aa8d913dad60 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:29.426554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 28/53] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:29:29 2022] GET /identity/v3/users/736ecd08572d41d59ba0fe2db80e313a => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:29:29.431855 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56554d2-c1fb-4790-93dd-aa8d913dad60 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:29.446635 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df4d1c-3b34-4b55-b84b-d089e6312d9e None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:29.484595 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f43e6-d98b-4b74-818d-5046bf0ce94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:29.485422 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f43e6-d98b-4b74-818d-5046bf0ce94b None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:29.485712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f43e6-d98b-4b74-818d-5046bf0ce94b None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:29.486012 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f43e6-d98b-4b74-818d-5046bf0ce94b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:29.487236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f43e6-d98b-4b74-818d-5046bf0ce94b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:29.488923 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f43e6-d98b-4b74-818d-5046bf0ce94b None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:29.490561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 26/54] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:29:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:29:29.496185 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f43e6-d98b-4b74-818d-5046bf0ce94b None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:29.536238 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56554d2-c1fb-4790-93dd-aa8d913dad60 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:29.572721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77f846-61eb-4fa5-b23c-884233ff6754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:29.573555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77f846-61eb-4fa5-b23c-884233ff6754 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:29.573851 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77f846-61eb-4fa5-b23c-884233ff6754 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:29.574140 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb77f846-61eb-4fa5-b23c-884233ff6754 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:29.575716 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77f846-61eb-4fa5-b23c-884233ff6754 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:29.578514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb77f846-61eb-4fa5-b23c-884233ff6754 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:29.580173 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 29/55] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:29:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:29:29.586112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77f846-61eb-4fa5-b23c-884233ff6754 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:29.602822 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f43e6-d98b-4b74-818d-5046bf0ce94b None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:29.644290 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1163f44-e1c1-42e0-b0f0-46541eb5459d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:29.645023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1163f44-e1c1-42e0-b0f0-46541eb5459d None admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:29.645322 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1163f44-e1c1-42e0-b0f0-46541eb5459d None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:29.645613 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1163f44-e1c1-42e0-b0f0-46541eb5459d None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:29.646348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1163f44-e1c1-42e0-b0f0-46541eb5459d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:29.647833 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1163f44-e1c1-42e0-b0f0-46541eb5459d None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:29.658318 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:29.663135 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:29.695985 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb77f846-61eb-4fa5-b23c-884233ff6754 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:29.730375 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491f9da-20f1-4dc9-9d65-5aafaf8ad927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:29.731032 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491f9da-20f1-4dc9-9d65-5aafaf8ad927 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:29.731321 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491f9da-20f1-4dc9-9d65-5aafaf8ad927 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:29.731604 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491f9da-20f1-4dc9-9d65-5aafaf8ad927 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:29.732439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491f9da-20f1-4dc9-9d65-5aafaf8ad927 None admin] RBAC: Adding query filter params (role.id=6505bbb129404436966ee59aa7a01d8e, scope.domain.id=default, user.id=736ecd08572d41d59ba0fe2db80e313a) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:29.732753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491f9da-20f1-4dc9-9d65-5aafaf8ad927 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:29.734468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491f9da-20f1-4dc9-9d65-5aafaf8ad927 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:29.740624 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 30/56] 10.222.0.13 () {56 vars in 1322 bytes} [Fri May 27 19:29:29 2022] GET /identity/v3/role_assignments?user.id=736ecd08572d41d59ba0fe2db80e313a&scope.domain.id=default&role.id=6505bbb129404436966ee59aa7a01d8e => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:29:29.875257 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 27/57] 10.222.0.13 () {60 vars in 1140 bytes} [Fri May 27 19:29:29 2022] POST /identity/v3/projects => generated 298 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:29:32.111518 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6db4b-e9c6-48b8-b31c-887d1f9a8ec5 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:32.112101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6db4b-e9c6-48b8-b31c-887d1f9a8ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:32.112101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6db4b-e9c6-48b8-b31c-887d1f9a8ec5 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:32.113502 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 31/58] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:32.118744 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315a6ac-d850-435d-8e82-028a43b403a9 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:32.119166 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315a6ac-d850-435d-8e82-028a43b403a9 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:32.119319 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7315a6ac-d850-435d-8e82-028a43b403a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:32.138245 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-7315a6ac-d850-435d-8e82-028a43b403a9 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:32.140141 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-7315a6ac-d850-435d-8e82-028a43b403a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:32.206834 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 28/59] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:32.231358 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6db4b-e9c6-48b8-b31c-887d1f9a8ec5 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:32.332975 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6db4b-e9c6-48b8-b31c-887d1f9a8ec5 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:32.362287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:32.366908 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96176af3-4bdf-44ce-9f82-48e89a0772b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:32.367700 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96176af3-4bdf-44ce-9f82-48e89a0772b7 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:32.367988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96176af3-4bdf-44ce-9f82-48e89a0772b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:32.368267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96176af3-4bdf-44ce-9f82-48e89a0772b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:32.369531 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96176af3-4bdf-44ce-9f82-48e89a0772b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:32.371005 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96176af3-4bdf-44ce-9f82-48e89a0772b7 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:32.372670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 32/60] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:29:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:32.378814 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7315a6ac-d850-435d-8e82-028a43b403a9 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:32.475547 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7315a6ac-d850-435d-8e82-028a43b403a9 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:32.499889 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:32.512368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1247b529-d8e9-41c7-a6c2-0f58d039607a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:32.512368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247b529-d8e9-41c7-a6c2-0f58d039607a None admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:32.512368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247b529-d8e9-41c7-a6c2-0f58d039607a None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:32.512669 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1247b529-d8e9-41c7-a6c2-0f58d039607a None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:32.514007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247b529-d8e9-41c7-a6c2-0f58d039607a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:32.516439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1247b529-d8e9-41c7-a6c2-0f58d039607a None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:32.619821 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 29/61] 10.222.0.13 () {60 vars in 1132 bytes} [Fri May 27 19:29:32 2022] POST /identity/v3/users => generated 275 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:29:36.201224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee0c53-e6d3-40c7-a9b8-8e16242588a2 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:36.201224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee0c53-e6d3-40c7-a9b8-8e16242588a2 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:36.201224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ee0c53-e6d3-40c7-a9b8-8e16242588a2 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:36.201224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 33/62] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:36.215135 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f2fb2-6a71-46aa-8a38-e7f6c1437822 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:36.215638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f2fb2-6a71-46aa-8a38-e7f6c1437822 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:36.215728 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f2fb2-6a71-46aa-8a38-e7f6c1437822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:36.225183 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31327cc-0c43-4342-be24-36bb568d3f35 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:36.225959 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31327cc-0c43-4342-be24-36bb568d3f35 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:36.226453 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31327cc-0c43-4342-be24-36bb568d3f35 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:36.228080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 34/63] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:36.238778 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64be6a5-ffcc-4424-9a77-534701870618 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:36.239425 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64be6a5-ffcc-4424-9a77-534701870618 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:36.239910 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64be6a5-ffcc-4424-9a77-534701870618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:36.246029 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-0f4f2fb2-6a71-46aa-8a38-e7f6c1437822 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:36.247462 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-0f4f2fb2-6a71-46aa-8a38-e7f6c1437822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:36.282381 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-c64be6a5-ffcc-4424-9a77-534701870618 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:36.283992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-c64be6a5-ffcc-4424-9a77-534701870618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:36.415097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 35/64] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 19:29:36.416921 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a261e-80cc-42da-86c5-735adbe07301 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:36.417477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a261e-80cc-42da-86c5-735adbe07301 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:36.418003 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a261e-80cc-42da-86c5-735adbe07301 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:36.418855 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 30/65] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:36.419765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 36/66] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:36.425963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce605fb6-0fb3-44c4-ba97-3dcd8e60dda5 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:36.426698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce605fb6-0fb3-44c4-ba97-3dcd8e60dda5 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:36.427259 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce605fb6-0fb3-44c4-ba97-3dcd8e60dda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:36.446701 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-ce605fb6-0fb3-44c4-ba97-3dcd8e60dda5 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:36.448247 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-ce605fb6-0fb3-44c4-ba97-3dcd8e60dda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:36.454695 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a261e-80cc-42da-86c5-735adbe07301 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:36.600631 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 31/67] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 19:29:36.602432 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce605fb6-0fb3-44c4-ba97-3dcd8e60dda5 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:36.617481 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a261e-80cc-42da-86c5-735adbe07301 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:36.665406 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:36.668931 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:36.678631 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172e2f6-a90f-4065-96bc-63fb4538608a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:36.689959 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172e2f6-a90f-4065-96bc-63fb4538608a None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:36.689959 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172e2f6-a90f-4065-96bc-63fb4538608a None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:36.689959 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172e2f6-a90f-4065-96bc-63fb4538608a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:36.695534 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2172e2f6-a90f-4065-96bc-63fb4538608a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:29:36.705798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-2172e2f6-a90f-4065-96bc-63fb4538608a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:29:36.705798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:36.705798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:36.705798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:36.705798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:36.705798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:36.705798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:36.705798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:36.705798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:36.705798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:36.705798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:36.705798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:36.705798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:29:36.705798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:29:36.705798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:29:36.705798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:29:36.706750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:29:36.706750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:29:36.706750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:29:36.706750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:29:36.706750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:36.706750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:36.706750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:36.706750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:36.706750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:36.706750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:36.706750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:36.706750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:29:36.706750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:36.706750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:36.706750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:36.706750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:36.707570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:36.707570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:36.707570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:36.707570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:36.707570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:29:36.707570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:29:36.707570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:29:36.707570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:29:36.707570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:29:36.707570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:29:36.707570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:29:36.707570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:29:36.710825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 37/68] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:29:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:29:36.712261 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172e2f6-a90f-4065-96bc-63fb4538608a None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:36.734585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce605fb6-0fb3-44c4-ba97-3dcd8e60dda5 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:36.795055 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f3c48-8c77-4cd7-8f9c-331858330a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:36.797672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f3c48-8c77-4cd7-8f9c-331858330a05 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:36.798401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f3c48-8c77-4cd7-8f9c-331858330a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:36.799478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906f3c48-8c77-4cd7-8f9c-331858330a05 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:36.815715 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906f3c48-8c77-4cd7-8f9c-331858330a05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:29:36.828400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-906f3c48-8c77-4cd7-8f9c-331858330a05 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:29:36.828400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:36.828400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:36.828400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:36.828400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:36.828400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:36.828400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:36.828400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:36.828400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:36.828400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:36.828400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:36.828400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:36.828400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:29:36.828400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:29:36.828400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:29:36.828400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:29:36.829839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:29:36.829839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:29:36.829839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:29:36.829839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:29:36.829839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:36.829839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:36.829839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:36.829839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:36.829839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:36.829839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:36.829839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:36.829839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:29:36.829839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:36.829839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:36.829839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:36.829839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:36.830997 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:36.830997 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:36.830997 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:36.830997 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:36.830997 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:29:36.830997 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:29:36.830997 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:29:36.830997 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:29:36.830997 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:29:36.830997 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:29:36.830997 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:29:36.830997 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:29:36.831590 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 32/69] 10.222.0.13 () {56 vars in 1114 bytes} [Fri May 27 19:29:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:29:36.832909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f3c48-8c77-4cd7-8f9c-331858330a05 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:36.861741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172e2f6-a90f-4065-96bc-63fb4538608a None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:36.908930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f3264-8182-4360-a3d2-2ddb55535def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:36.910660 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f3264-8182-4360-a3d2-2ddb55535def None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:36.911409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f3264-8182-4360-a3d2-2ddb55535def None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:36.911891 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f3264-8182-4360-a3d2-2ddb55535def None admin] PATH_INFO: `/v3/roles/member` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:36.925442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f3264-8182-4360-a3d2-2ddb55535def None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:29:36.933355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-f31f3264-8182-4360-a3d2-2ddb55535def None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 19:29:36.933355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:36.933355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:36.933355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:36.933355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:36.933355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:36.933355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:36.933355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:36.933355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:36.933355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:36.933355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:36.933355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:36.933355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:29:36.933355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:29:36.933355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:29:36.933355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:29:36.934257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:29:36.934257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:29:36.934257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:29:36.934257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:29:36.934257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:36.934257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:36.934257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:36.934257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:36.934257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:36.934257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:36.934257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:36.934257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:29:36.934257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:36.934257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:36.934257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:36.934257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:36.935441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:36.935441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:36.935441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:36.935441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:36.935441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:29:36.935441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:29:36.935441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:29:36.935441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:29:36.935441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:29:36.935441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:29:36.935441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 19:29:36.935441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:29:36.935441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 38/70] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:29:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:29:36.935441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f3264-8182-4360-a3d2-2ddb55535def None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:36.955205 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:36.958957 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906f3c48-8c77-4cd7-8f9c-331858330a05 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:36.995651 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b96474-b03e-4129-b58f-5469fc795425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:36.996425 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b96474-b03e-4129-b58f-5469fc795425 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:36.996715 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b96474-b03e-4129-b58f-5469fc795425 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:36.997023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b96474-b03e-4129-b58f-5469fc795425 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:36.997783 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b96474-b03e-4129-b58f-5469fc795425 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:36.998124 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b96474-b03e-4129-b58f-5469fc795425 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:37.000114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b96474-b03e-4129-b58f-5469fc795425 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:37.011871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:37.013004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 33/71] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:29:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:37.014317 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba5588-1321-4303-8d27-7a482f36db5f None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:37.014608 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba5588-1321-4303-8d27-7a482f36db5f None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:37.014891 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba5588-1321-4303-8d27-7a482f36db5f None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:37.016298 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 34/72] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:37.017922 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba5588-1321-4303-8d27-7a482f36db5f None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:37.068902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f3264-8182-4360-a3d2-2ddb55535def None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:37.104056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cea0c8-3fca-4fb6-84e5-98d2d3a915fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:37.104842 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cea0c8-3fca-4fb6-84e5-98d2d3a915fe None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:37.105220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cea0c8-3fca-4fb6-84e5-98d2d3a915fe None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:37.105563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cea0c8-3fca-4fb6-84e5-98d2d3a915fe None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:37.106382 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cea0c8-3fca-4fb6-84e5-98d2d3a915fe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:37.106792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cea0c8-3fca-4fb6-84e5-98d2d3a915fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:37.108459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cea0c8-3fca-4fb6-84e5-98d2d3a915fe None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:37.117335 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 39/73] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:29:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:37.118782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da4911-906c-4cff-b4f6-ec3eecaf372f None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:37.119084 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da4911-906c-4cff-b4f6-ec3eecaf372f None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:37.119366 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da4911-906c-4cff-b4f6-ec3eecaf372f None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:37.120696 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 40/74] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:37.121432 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da4911-906c-4cff-b4f6-ec3eecaf372f None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:37.139463 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba5588-1321-4303-8d27-7a482f36db5f None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:37.189860 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e1089-c5f5-49d2-b830-7d9a8b00807d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:37.190544 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e1089-c5f5-49d2-b830-7d9a8b00807d None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:37.190837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e1089-c5f5-49d2-b830-7d9a8b00807d None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:37.191125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e1089-c5f5-49d2-b830-7d9a8b00807d None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:37.191898 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e1089-c5f5-49d2-b830-7d9a8b00807d None admin] RBAC: Adding query filter params (name=member) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:37.192206 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e1089-c5f5-49d2-b830-7d9a8b00807d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:37.193653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e1089-c5f5-49d2-b830-7d9a8b00807d None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:37.207841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 35/75] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:29:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:37.207841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec6ad7-364a-478d-a3ad-b94a573a53f0 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:37.207841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec6ad7-364a-478d-a3ad-b94a573a53f0 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:37.207841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec6ad7-364a-478d-a3ad-b94a573a53f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:37.225983 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-24ec6ad7-364a-478d-a3ad-b94a573a53f0 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:37.227611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-24ec6ad7-364a-478d-a3ad-b94a573a53f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:37.245039 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da4911-906c-4cff-b4f6-ec3eecaf372f None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:37.248076 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:37.261387 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:37.285531 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38336588-3947-4b14-a47a-4b668d711064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:37.286420 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38336588-3947-4b14-a47a-4b668d711064 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:37.286708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38336588-3947-4b14-a47a-4b668d711064 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:37.286990 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38336588-3947-4b14-a47a-4b668d711064 None admin] PATH_INFO: `/v3/users/736ecd08572d41d59ba0fe2db80e313a` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:37.288229 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38336588-3947-4b14-a47a-4b668d711064 None admin] RBAC: Authorizing `identity:get_user(user_id=736ecd08572d41d59ba0fe2db80e313a)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:37.289821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38336588-3947-4b14-a47a-4b668d711064 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:37.291468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 41/76] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:29:37 2022] GET /identity/v3/users/736ecd08572d41d59ba0fe2db80e313a => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:37.292313 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38336588-3947-4b14-a47a-4b668d711064 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:37.314639 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 36/77] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:37.315943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54112f2-8ace-4290-b03b-c74a671321ae None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:37.316230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54112f2-8ace-4290-b03b-c74a671321ae None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:37.316509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54112f2-8ace-4290-b03b-c74a671321ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:37.333760 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-c54112f2-8ace-4290-b03b-c74a671321ae None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:37.335037 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-c54112f2-8ace-4290-b03b-c74a671321ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:37.413783 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38336588-3947-4b14-a47a-4b668d711064 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:37.419097 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 37/78] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:37.419799 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54112f2-8ace-4290-b03b-c74a671321ae None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:37.450951 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6078c-a886-4e8f-bdef-b3d3eea13d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:37.451655 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6078c-a886-4e8f-bdef-b3d3eea13d89 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:37.451940 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6078c-a886-4e8f-bdef-b3d3eea13d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:37.452226 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6078c-a886-4e8f-bdef-b3d3eea13d89 None admin] PATH_INFO: `/v3/users/02e527c314c64f03bbc6d51e8e772a09` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:37.466651 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6078c-a886-4e8f-bdef-b3d3eea13d89 None admin] RBAC: Authorizing `identity:get_user(user_id=02e527c314c64f03bbc6d51e8e772a09)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:37.469072 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6078c-a886-4e8f-bdef-b3d3eea13d89 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:37.470896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 42/79] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:29:37 2022] GET /identity/v3/users/02e527c314c64f03bbc6d51e8e772a09 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:37.471757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6078c-a886-4e8f-bdef-b3d3eea13d89 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:37.517526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:37.524840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:37.537436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54112f2-8ace-4290-b03b-c74a671321ae None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:37.571956 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff72189-703c-45bd-a688-efa269bf0baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:37.572850 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff72189-703c-45bd-a688-efa269bf0baa None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:37.573168 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff72189-703c-45bd-a688-efa269bf0baa None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:37.573454 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff72189-703c-45bd-a688-efa269bf0baa None admin] PATH_INFO: `/v3/users/02e527c314c64f03bbc6d51e8e772a09` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:37.574683 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff72189-703c-45bd-a688-efa269bf0baa None admin] RBAC: Authorizing `identity:get_user(user_id=02e527c314c64f03bbc6d51e8e772a09)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:37.576140 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff72189-703c-45bd-a688-efa269bf0baa None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:37.578547 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 38/80] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:29:37 2022] GET /identity/v3/users/02e527c314c64f03bbc6d51e8e772a09 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:37.587310 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff72189-703c-45bd-a688-efa269bf0baa None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:37.598532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6078c-a886-4e8f-bdef-b3d3eea13d89 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:37.635515 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d326c51-f8fa-4b8f-8a1c-e221af1f0a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:37.636199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d326c51-f8fa-4b8f-8a1c-e221af1f0a5d None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:37.636466 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d326c51-f8fa-4b8f-8a1c-e221af1f0a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:37.636776 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d326c51-f8fa-4b8f-8a1c-e221af1f0a5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:37.699313 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff72189-703c-45bd-a688-efa269bf0baa None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:37.711621 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d326c51-f8fa-4b8f-8a1c-e221af1f0a5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:37.713405 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d326c51-f8fa-4b8f-8a1c-e221af1f0a5d None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:37.746882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1616d4-fad0-4f51-91ff-6565e7cd37f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:37.747733 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1616d4-fad0-4f51-91ff-6565e7cd37f4 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:37.748024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1616d4-fad0-4f51-91ff-6565e7cd37f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:37.748307 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1616d4-fad0-4f51-91ff-6565e7cd37f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:37.759251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:37.760226 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1616d4-fad0-4f51-91ff-6565e7cd37f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:29:37.768875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-bb1616d4-fad0-4f51-91ff-6565e7cd37f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 19:29:37.768875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:37.768875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:37.768875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:37.768875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:37.768875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:37.768875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:37.768875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:37.768875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:37.768875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:37.768875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:37.768875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:37.768875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:29:37.768875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:29:37.768875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:29:37.768875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:29:37.770754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:29:37.770754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:29:37.770754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:29:37.770754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:29:37.770754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:37.770754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:37.770754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:37.770754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:37.770754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:37.770754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:37.770754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:37.770754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:29:37.770754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:37.770754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:37.770754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:37.770754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:37.771618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:37.771618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:37.771618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:37.771618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:37.771618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:29:37.771618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:29:37.771618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:29:37.771618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:29:37.771618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:29:37.771618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:29:37.771618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 19:29:37.771618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:29:37.771618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 39/81] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:29:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:29:37.771618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1616d4-fad0-4f51-91ff-6565e7cd37f4 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:37.784974 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:37.829505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 43/82] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:29:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:29:37.830432 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d326c51-f8fa-4b8f-8a1c-e221af1f0a5d None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:37.904685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1616d4-fad0-4f51-91ff-6565e7cd37f4 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:37.940424 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7905737-f6e9-43f6-a7f4-46ac0b1252c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:37.941250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7905737-f6e9-43f6-a7f4-46ac0b1252c6 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:37.941542 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7905737-f6e9-43f6-a7f4-46ac0b1252c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:37.941820 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7905737-f6e9-43f6-a7f4-46ac0b1252c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:37.943012 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7905737-f6e9-43f6-a7f4-46ac0b1252c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:37.944496 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7905737-f6e9-43f6-a7f4-46ac0b1252c6 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:37.946350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 40/83] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:29:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:37.947121 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7905737-f6e9-43f6-a7f4-46ac0b1252c6 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:37.952035 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d326c51-f8fa-4b8f-8a1c-e221af1f0a5d None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:37.998051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbdf822-6e53-4c6e-9d33-73184ddeb3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:37.998710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbdf822-6e53-4c6e-9d33-73184ddeb3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:37.999000 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbdf822-6e53-4c6e-9d33-73184ddeb3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:37.999281 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbdf822-6e53-4c6e-9d33-73184ddeb3b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:38.008554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:38.037755 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:38.064417 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7905737-f6e9-43f6-a7f4-46ac0b1252c6 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:38.074052 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbdf822-6e53-4c6e-9d33-73184ddeb3b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:38.075590 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbdf822-6e53-4c6e-9d33-73184ddeb3b8 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:38.102293 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199cf2c-2fd7-4851-be19-d3f4cb5523df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:38.103001 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199cf2c-2fd7-4851-be19-d3f4cb5523df None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:38.103282 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199cf2c-2fd7-4851-be19-d3f4cb5523df None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:38.103602 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199cf2c-2fd7-4851-be19-d3f4cb5523df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:38.184193 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a199cf2c-2fd7-4851-be19-d3f4cb5523df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:38.185727 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a199cf2c-2fd7-4851-be19-d3f4cb5523df None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:38.190998 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 44/84] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:29:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:29:38.191861 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbdf822-6e53-4c6e-9d33-73184ddeb3b8 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:38.233544 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:38.277085 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:38.297798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbdf822-6e53-4c6e-9d33-73184ddeb3b8 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:38.300069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 41/85] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:29:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:29:38.304787 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199cf2c-2fd7-4851-be19-d3f4cb5523df None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:38.335854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97965c8b-8449-4cb2-8f68-edd298eb2e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:38.336536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97965c8b-8449-4cb2-8f68-edd298eb2e2e None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:38.336828 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97965c8b-8449-4cb2-8f68-edd298eb2e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:38.337237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97965c8b-8449-4cb2-8f68-edd298eb2e2e None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:38.337956 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97965c8b-8449-4cb2-8f68-edd298eb2e2e None admin] RBAC: Adding query filter params (name=member) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:38.338272 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97965c8b-8449-4cb2-8f68-edd298eb2e2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:38.339897 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97965c8b-8449-4cb2-8f68-edd298eb2e2e None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:38.350155 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 45/86] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:29:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:38.350991 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97965c8b-8449-4cb2-8f68-edd298eb2e2e None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:38.410497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199cf2c-2fd7-4851-be19-d3f4cb5523df None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:38.445108 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c6a47-f484-4889-84b9-5d2c6176c831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:38.445982 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c6a47-f484-4889-84b9-5d2c6176c831 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:38.446272 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c6a47-f484-4889-84b9-5d2c6176c831 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:38.446560 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3c6a47-f484-4889-84b9-5d2c6176c831 None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:38.450589 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c6a47-f484-4889-84b9-5d2c6176c831 None admin] RBAC: Authorizing `identity:get_project(project_id=b30f38a32f6d4982bd829828ccac6fa7)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:38.450736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:38.452057 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3c6a47-f484-4889-84b9-5d2c6176c831 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:38.454350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 42/87] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:29:38 2022] GET /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:38.466595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97965c8b-8449-4cb2-8f68-edd298eb2e2e None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:38.466729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c6a47-f484-4889-84b9-5d2c6176c831 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:38.492920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686ea16-6f0e-43bb-8981-7e9165c0edcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:38.493591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686ea16-6f0e-43bb-8981-7e9165c0edcf None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:38.493875 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686ea16-6f0e-43bb-8981-7e9165c0edcf None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:38.494157 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e686ea16-6f0e-43bb-8981-7e9165c0edcf None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:38.494871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686ea16-6f0e-43bb-8981-7e9165c0edcf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:38.496354 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e686ea16-6f0e-43bb-8981-7e9165c0edcf None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:38.510709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:38.574649 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3c6a47-f484-4889-84b9-5d2c6176c831 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:38.608885 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 46/88] 10.222.0.13 () {60 vars in 1132 bytes} [Fri May 27 19:29:38 2022] POST /identity/v3/users => generated 289 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:29:38.609866 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686ea16-6f0e-43bb-8981-7e9165c0edcf None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:38.616799 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027929a-7149-4955-aaac-fcd0bd2533a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:38.617746 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027929a-7149-4955-aaac-fcd0bd2533a8 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:38.618069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027929a-7149-4955-aaac-fcd0bd2533a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:38.618354 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027929a-7149-4955-aaac-fcd0bd2533a8 None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:38.619686 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027929a-7149-4955-aaac-fcd0bd2533a8 None admin] RBAC: Authorizing `identity:get_project(project_id=b30f38a32f6d4982bd829828ccac6fa7)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:38.621237 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027929a-7149-4955-aaac-fcd0bd2533a8 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:38.623132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 43/89] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:29:38 2022] GET /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:38.624993 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027929a-7149-4955-aaac-fcd0bd2533a8 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:38.762919 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e686ea16-6f0e-43bb-8981-7e9165c0edcf None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:38.814843 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027929a-7149-4955-aaac-fcd0bd2533a8 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:38.817294 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf63ecf-8d4c-45a0-8dc0-05f06d80f3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:38.818480 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf63ecf-8d4c-45a0-8dc0-05f06d80f3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:38.818771 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf63ecf-8d4c-45a0-8dc0-05f06d80f3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:38.820003 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf63ecf-8d4c-45a0-8dc0-05f06d80f3a9 None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:38.821361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf63ecf-8d4c-45a0-8dc0-05f06d80f3a9 None admin] RBAC: Authorizing `identity:get_project(project_id=b30f38a32f6d4982bd829828ccac6fa7)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:38.822896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf63ecf-8d4c-45a0-8dc0-05f06d80f3a9 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:38.824785 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 47/90] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:29:38 2022] GET /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7 => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:38.827298 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf63ecf-8d4c-45a0-8dc0-05f06d80f3a9 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:38.849266 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:38.869469 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:38.889063 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02983924-6fbb-462e-a4e1-b7ba6befac98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:38.891280 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02983924-6fbb-462e-a4e1-b7ba6befac98 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:38.892494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02983924-6fbb-462e-a4e1-b7ba6befac98 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:38.892494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02983924-6fbb-462e-a4e1-b7ba6befac98 None admin] PATH_INFO: `/v3/users/02e527c314c64f03bbc6d51e8e772a09` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:38.893984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02983924-6fbb-462e-a4e1-b7ba6befac98 None admin] RBAC: Authorizing `identity:get_user(user_id=02e527c314c64f03bbc6d51e8e772a09)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:38.897325 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02983924-6fbb-462e-a4e1-b7ba6befac98 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:38.898867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 44/91] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:29:38 2022] GET /identity/v3/users/02e527c314c64f03bbc6d51e8e772a09 => generated 275 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:38.900003 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02983924-6fbb-462e-a4e1-b7ba6befac98 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:38.962715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf63ecf-8d4c-45a0-8dc0-05f06d80f3a9 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:39.019500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8280f62-418c-44aa-a3f4-d3b254948b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:39.020572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8280f62-418c-44aa-a3f4-d3b254948b15 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:39.020868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8280f62-418c-44aa-a3f4-d3b254948b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:39.021179 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8280f62-418c-44aa-a3f4-d3b254948b15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:39.022228 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8280f62-418c-44aa-a3f4-d3b254948b15 None admin] RBAC: Adding query filter params (role.id=6505bbb129404436966ee59aa7a01d8e, scope.project.id=b30f38a32f6d4982bd829828ccac6fa7, user.id=736ecd08572d41d59ba0fe2db80e313a) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:39.022551 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8280f62-418c-44aa-a3f4-d3b254948b15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:39.024083 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8280f62-418c-44aa-a3f4-d3b254948b15 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:39.033014 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 48/92] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:29:38 2022] GET /identity/v3/role_assignments?user.id=736ecd08572d41d59ba0fe2db80e313a&scope.project.id=b30f38a32f6d4982bd829828ccac6fa7&role.id=6505bbb129404436966ee59aa7a01d8e => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:39.033014 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8280f62-418c-44aa-a3f4-d3b254948b15 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:39.037891 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02983924-6fbb-462e-a4e1-b7ba6befac98 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:39.083031 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b06e8c-0a80-4e32-8fcc-ec529fbead5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:39.083638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b06e8c-0a80-4e32-8fcc-ec529fbead5f None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:39.083926 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b06e8c-0a80-4e32-8fcc-ec529fbead5f None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:39.084245 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b06e8c-0a80-4e32-8fcc-ec529fbead5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:39.085284 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b06e8c-0a80-4e32-8fcc-ec529fbead5f None admin] RBAC: Adding query filter params (role.id=eda4ac05bc364224a7f319e7ded54073, scope.project.id=b30f38a32f6d4982bd829828ccac6fa7, user.id=02e527c314c64f03bbc6d51e8e772a09) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:39.085604 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b06e8c-0a80-4e32-8fcc-ec529fbead5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:39.087252 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b06e8c-0a80-4e32-8fcc-ec529fbead5f None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:39.098004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 45/93] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:29:38 2022] GET /identity/v3/role_assignments?user.id=02e527c314c64f03bbc6d51e8e772a09&scope.project.id=b30f38a32f6d4982bd829828ccac6fa7&role.id=eda4ac05bc364224a7f319e7ded54073 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:39.098746 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b06e8c-0a80-4e32-8fcc-ec529fbead5f None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:39.165332 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:39.172423 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:39.189999 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8280f62-418c-44aa-a3f4-d3b254948b15 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:39.236834 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53139df8-a38e-4e30-9cac-e0b7aa9a97ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:39.238657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53139df8-a38e-4e30-9cac-e0b7aa9a97ea None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:39.239143 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53139df8-a38e-4e30-9cac-e0b7aa9a97ea None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:39.239568 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53139df8-a38e-4e30-9cac-e0b7aa9a97ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:39.240659 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53139df8-a38e-4e30-9cac-e0b7aa9a97ea None admin] RBAC: Adding query filter params (role.id=cb43a575e47147c799eec2f362f4fe0e, scope.project.id=b30f38a32f6d4982bd829828ccac6fa7, user.id=02e527c314c64f03bbc6d51e8e772a09) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:39.241170 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53139df8-a38e-4e30-9cac-e0b7aa9a97ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:39.243274 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53139df8-a38e-4e30-9cac-e0b7aa9a97ea None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:39.251947 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 49/94] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:29:39 2022] GET /identity/v3/role_assignments?user.id=02e527c314c64f03bbc6d51e8e772a09&scope.project.id=b30f38a32f6d4982bd829828ccac6fa7&role.id=cb43a575e47147c799eec2f362f4fe0e => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:39.281275 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b06e8c-0a80-4e32-8fcc-ec529fbead5f None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:39.328727 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49054609-b38d-48fb-a8fd-9f34832e0eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:39.329464 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49054609-b38d-48fb-a8fd-9f34832e0eed None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:39.329757 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49054609-b38d-48fb-a8fd-9f34832e0eed None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:39.330046 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49054609-b38d-48fb-a8fd-9f34832e0eed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:39.412820 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49054609-b38d-48fb-a8fd-9f34832e0eed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:39.414346 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49054609-b38d-48fb-a8fd-9f34832e0eed None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:39.470372 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:39.528472 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 46/95] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:29:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:29:39.535144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53139df8-a38e-4e30-9cac-e0b7aa9a97ea None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:39.670088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53139df8-a38e-4e30-9cac-e0b7aa9a97ea None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:39.719773 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f543f5f-293d-4761-8d55-2dd64fe7871d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:39.721380 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f543f5f-293d-4761-8d55-2dd64fe7871d None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:39.721380 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f543f5f-293d-4761-8d55-2dd64fe7871d None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:39.721380 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f543f5f-293d-4761-8d55-2dd64fe7871d None admin] PATH_INFO: `/v3/projects/c2c64289866e4b5d949e9a8c4012f2c8` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:39.725719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f543f5f-293d-4761-8d55-2dd64fe7871d None admin] RBAC: Authorizing `identity:get_project(project_id=c2c64289866e4b5d949e9a8c4012f2c8)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:39.727242 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f543f5f-293d-4761-8d55-2dd64fe7871d None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:39.729062 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 50/96] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:29:39 2022] GET /identity/v3/projects/c2c64289866e4b5d949e9a8c4012f2c8 => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:39.735415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49054609-b38d-48fb-a8fd-9f34832e0eed None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:39.851977 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49054609-b38d-48fb-a8fd-9f34832e0eed None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:39.893792 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779b5387-11b1-4c74-8300-1f3b44bcddba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:39.894462 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b5387-11b1-4c74-8300-1f3b44bcddba None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:39.894752 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b5387-11b1-4c74-8300-1f3b44bcddba None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:39.895032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b5387-11b1-4c74-8300-1f3b44bcddba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:39.895886 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b5387-11b1-4c74-8300-1f3b44bcddba None admin] RBAC: Adding query filter params (role.id=cb43a575e47147c799eec2f362f4fe0e, scope.project.id=c2c64289866e4b5d949e9a8c4012f2c8, user.id=02e527c314c64f03bbc6d51e8e772a09) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:39.896203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b5387-11b1-4c74-8300-1f3b44bcddba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:39.897811 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779b5387-11b1-4c74-8300-1f3b44bcddba None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:39.904887 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 47/97] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:29:39 2022] GET /identity/v3/role_assignments?user.id=02e527c314c64f03bbc6d51e8e772a09&scope.project.id=c2c64289866e4b5d949e9a8c4012f2c8&role.id=cb43a575e47147c799eec2f362f4fe0e => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:29:42.127719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90225a55-ed5e-4dd7-a714-fb78ebc3d42e None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:42.128235 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90225a55-ed5e-4dd7-a714-fb78ebc3d42e None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:42.128308 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90225a55-ed5e-4dd7-a714-fb78ebc3d42e None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:42.129941 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 51/98] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:42.141352 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0e40d-afa0-4098-9481-b92d6defab84 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:42.141852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0e40d-afa0-4098-9481-b92d6defab84 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:42.141929 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0e40d-afa0-4098-9481-b92d6defab84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:42.156743 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55684dea-2658-456a-9b89-a551fa0e2232 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:42.157064 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55684dea-2658-456a-9b89-a551fa0e2232 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:42.157351 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55684dea-2658-456a-9b89-a551fa0e2232 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:42.158957 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 52/99] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:42.170677 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e907a4c-25fd-4a74-aeb5-c4a06a178fc7 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:42.171583 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e907a4c-25fd-4a74-aeb5-c4a06a178fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:42.171882 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e907a4c-25fd-4a74-aeb5-c4a06a178fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:42.181843 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-18f0e40d-afa0-4098-9481-b92d6defab84 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:42.188327 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-18f0e40d-afa0-4098-9481-b92d6defab84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:42.193896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:42.206074 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-9e907a4c-25fd-4a74-aeb5-c4a06a178fc7 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:42.207294 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-9e907a4c-25fd-4a74-aeb5-c4a06a178fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:42.240013 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:42.327889 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 48/100] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:42.329091 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93571d77-11e2-4ff0-a314-444dadc2ac84 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:42.329387 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93571d77-11e2-4ff0-a314-444dadc2ac84 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:42.329669 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93571d77-11e2-4ff0-a314-444dadc2ac84 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:42.330951 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 49/101] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:42.336944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa03f9-5e92-4fae-9f1d-4f6707b04ee3 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:42.337317 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa03f9-5e92-4fae-9f1d-4f6707b04ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:42.337563 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa03f9-5e92-4fae-9f1d-4f6707b04ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:42.341025 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 53/102] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 27 19:29:42.366854 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-7daa03f9-5e92-4fae-9f1d-4f6707b04ee3 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:42.368038 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-7daa03f9-5e92-4fae-9f1d-4f6707b04ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:42.414157 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e907a4c-25fd-4a74-aeb5-c4a06a178fc7 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:42.451607 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 50/103] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:42.453071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa03f9-5e92-4fae-9f1d-4f6707b04ee3 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:42.604425 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:42.618560 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e907a4c-25fd-4a74-aeb5-c4a06a178fc7 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:42.668136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa03f9-5e92-4fae-9f1d-4f6707b04ee3 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:42.706220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427a255-b386-47bd-bb30-84022982803e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:42.707411 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427a255-b386-47bd-bb30-84022982803e None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:42.707895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427a255-b386-47bd-bb30-84022982803e None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:42.708351 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427a255-b386-47bd-bb30-84022982803e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:42.734998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a9f1f-5ee5-4c69-8fd7-f38bb1212eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:42.736042 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a9f1f-5ee5-4c69-8fd7-f38bb1212eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:42.736475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a9f1f-5ee5-4c69-8fd7-f38bb1212eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:42.736909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155a9f1f-5ee5-4c69-8fd7-f38bb1212eb7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:42.737586 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4427a255-b386-47bd-bb30-84022982803e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:29:42.738420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155a9f1f-5ee5-4c69-8fd7-f38bb1212eb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:42.740883 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155a9f1f-5ee5-4c69-8fd7-f38bb1212eb7 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:42.743080 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 51/104] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:29:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:42.744006 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a9f1f-5ee5-4c69-8fd7-f38bb1212eb7 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:42.750097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-4427a255-b386-47bd-bb30-84022982803e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 19:29:42.750097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:42.750097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:42.750097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:42.750097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:42.750097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:42.750097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:42.750097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:42.750097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:42.750097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:42.750097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:42.750097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:42.750097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:29:42.750097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:29:42.750097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:29:42.750097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:29:42.751167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:29:42.751167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:29:42.751167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:29:42.751167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:29:42.751167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:42.751167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:42.751167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:42.751167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:42.751167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:42.751167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:42.751167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:42.751167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:29:42.751167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:42.751167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:42.751167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:42.751167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:42.752042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:42.752042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:42.752042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:42.752042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:42.752042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:29:42.752042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:29:42.752042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:29:42.752042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:29:42.752042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:29:42.752042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:29:42.752042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 19:29:42.752042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:29:42.752042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 54/105] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:29:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:29:42.752671 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427a255-b386-47bd-bb30-84022982803e None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:42.776300 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:42.863864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155a9f1f-5ee5-4c69-8fd7-f38bb1212eb7 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:42.900914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315ed8a-7f9b-404c-a3ed-bc97392cfd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:42.902185 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315ed8a-7f9b-404c-a3ed-bc97392cfd6e None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:42.902466 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315ed8a-7f9b-404c-a3ed-bc97392cfd6e None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:42.903235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315ed8a-7f9b-404c-a3ed-bc97392cfd6e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:42.910439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:42.914008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a315ed8a-7f9b-404c-a3ed-bc97392cfd6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:29:42.928932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-a315ed8a-7f9b-404c-a3ed-bc97392cfd6e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 19:29:42.928932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:42.928932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:42.928932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:42.928932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:42.928932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:42.928932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:42.928932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:42.928932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:42.928932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:42.928932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:42.928932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:42.928932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:29:42.928932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:29:42.928932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:29:42.928932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:29:42.929857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:29:42.929857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:29:42.929857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:29:42.929857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:29:42.929857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:42.929857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:42.929857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:42.929857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:42.929857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:42.929857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:42.929857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:42.929857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:29:42.929857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:42.929857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:42.929857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:42.929857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:42.930634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:42.930634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:42.930634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:42.930634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:42.930634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:29:42.930634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:29:42.930634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:29:42.930634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:29:42.930634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:29:42.930634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:29:42.930634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 19:29:42.930634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:29:42.930634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 52/106] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:29:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:29:42.930634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315ed8a-7f9b-404c-a3ed-bc97392cfd6e None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:42.947807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427a255-b386-47bd-bb30-84022982803e None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:42.990263 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0751bfb-0acc-4429-a73b-afe010701002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:42.990959 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0751bfb-0acc-4429-a73b-afe010701002 None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:42.991249 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0751bfb-0acc-4429-a73b-afe010701002 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:42.991536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0751bfb-0acc-4429-a73b-afe010701002 None admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:42.992283 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0751bfb-0acc-4429-a73b-afe010701002 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:42.993851 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0751bfb-0acc-4429-a73b-afe010701002 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:43.035593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315ed8a-7f9b-404c-a3ed-bc97392cfd6e None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:43.069476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e57fa-c3a1-4e5c-802a-0663113338d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:43.070150 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e57fa-c3a1-4e5c-802a-0663113338d6 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:43.070441 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e57fa-c3a1-4e5c-802a-0663113338d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:43.070725 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e57fa-c3a1-4e5c-802a-0663113338d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:43.073279 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e57fa-c3a1-4e5c-802a-0663113338d6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:43.073600 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e57fa-c3a1-4e5c-802a-0663113338d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:43.075011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304e57fa-c3a1-4e5c-802a-0663113338d6 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:43.088985 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 55/107] 10.222.0.13 () {60 vars in 1141 bytes} [Fri May 27 19:29:42 2022] POST /identity/v3/projects => generated 302 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:29:43.088985 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f43d07-9c60-42d7-ad46-309936e28084 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:43.088985 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f43d07-9c60-42d7-ad46-309936e28084 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:43.088985 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f43d07-9c60-42d7-ad46-309936e28084 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:43.089354 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 53/108] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:29:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:43.089354 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e57fa-c3a1-4e5c-802a-0663113338d6 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:43.089991 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 56/109] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:43.091752 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff9900-b2ca-43fe-957e-c6bb822b16e7 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:43.092362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff9900-b2ca-43fe-957e-c6bb822b16e7 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:43.092877 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff9900-b2ca-43fe-957e-c6bb822b16e7 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:43.095990 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 57/110] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:43.096869 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff9900-b2ca-43fe-957e-c6bb822b16e7 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:43.164612 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:43.190484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:43.213400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e57fa-c3a1-4e5c-802a-0663113338d6 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:43.239622 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff9900-b2ca-43fe-957e-c6bb822b16e7 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:43.251978 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4294f767-f09e-4231-899f-7df6603af1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:43.252665 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294f767-f09e-4231-899f-7df6603af1af None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:43.252978 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294f767-f09e-4231-899f-7df6603af1af None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:43.253269 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4294f767-f09e-4231-899f-7df6603af1af None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:43.254038 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294f767-f09e-4231-899f-7df6603af1af None admin] RBAC: Adding query filter params (name=member) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:43.254363 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294f767-f09e-4231-899f-7df6603af1af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:43.255757 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4294f767-f09e-4231-899f-7df6603af1af None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:43.270203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 54/111] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:29:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:43.270203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfeb451-ec8c-451c-956f-3bc463202141 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:43.270203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfeb451-ec8c-451c-956f-3bc463202141 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:43.270203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfeb451-ec8c-451c-956f-3bc463202141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:43.284698 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d91ffec-41ef-407a-834e-18a56ddab708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:43.285622 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91ffec-41ef-407a-834e-18a56ddab708 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:43.285933 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91ffec-41ef-407a-834e-18a56ddab708 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:43.286212 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91ffec-41ef-407a-834e-18a56ddab708 None admin] PATH_INFO: `/v3/users/e00abb1711cb497187586e15b695c7d0` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:43.286329 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-dcfeb451-ec8c-451c-956f-3bc463202141 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:43.287470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-dcfeb451-ec8c-451c-956f-3bc463202141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:43.302713 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d91ffec-41ef-407a-834e-18a56ddab708 None admin] RBAC: Authorizing `identity:get_user(user_id=e00abb1711cb497187586e15b695c7d0)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:43.304812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d91ffec-41ef-407a-834e-18a56ddab708 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:43.306819 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 58/112] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:29:43 2022] GET /identity/v3/users/e00abb1711cb497187586e15b695c7d0 => generated 289 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:43.308075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f2d1d-d7f4-4211-a23d-2823f599def2 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:43.308365 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f2d1d-d7f4-4211-a23d-2823f599def2 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:43.308647 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f2d1d-d7f4-4211-a23d-2823f599def2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:43.329091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-e08f2d1d-d7f4-4211-a23d-2823f599def2 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:43.330329 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-e08f2d1d-d7f4-4211-a23d-2823f599def2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:43.370268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 55/113] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:43.371938 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c2d8d-d1ec-4222-b8da-ece95322f2fa None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:43.372424 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c2d8d-d1ec-4222-b8da-ece95322f2fa None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:43.372896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c2d8d-d1ec-4222-b8da-ece95322f2fa None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:43.374751 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 56/114] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:43.375679 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c2d8d-d1ec-4222-b8da-ece95322f2fa None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:43.414485 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 59/115] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:43.415284 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f2d1d-d7f4-4211-a23d-2823f599def2 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:43.451775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:43.471624 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:43.505615 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c2d8d-d1ec-4222-b8da-ece95322f2fa None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:43.531428 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f2d1d-d7f4-4211-a23d-2823f599def2 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:43.547243 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e7ab9-2d7f-4182-9492-d659aea96dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:43.548161 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e7ab9-2d7f-4182-9492-d659aea96dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:43.548416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e7ab9-2d7f-4182-9492-d659aea96dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:43.548689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e7ab9-2d7f-4182-9492-d659aea96dc0 None admin] PATH_INFO: `/v3/users/02e527c314c64f03bbc6d51e8e772a09` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:43.549914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e7ab9-2d7f-4182-9492-d659aea96dc0 None admin] RBAC: Authorizing `identity:get_user(user_id=02e527c314c64f03bbc6d51e8e772a09)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:43.551374 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e7ab9-2d7f-4182-9492-d659aea96dc0 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:43.553331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 57/116] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:29:43 2022] GET /identity/v3/users/02e527c314c64f03bbc6d51e8e772a09 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:43.554271 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248803f-b1e4-45d5-a900-536943e3f5d5 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:43.554557 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248803f-b1e4-45d5-a900-536943e3f5d5 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:43.554833 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248803f-b1e4-45d5-a900-536943e3f5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:43.572052 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-8248803f-b1e4-45d5-a900-536943e3f5d5 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:43.573254 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-8248803f-b1e4-45d5-a900-536943e3f5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:43.573844 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1e266-459c-45e4-a386-194750c4883d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:43.574507 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1e266-459c-45e4-a386-194750c4883d None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:43.574800 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1e266-459c-45e4-a386-194750c4883d None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:43.575088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e1e266-459c-45e4-a386-194750c4883d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:43.646095 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 58/117] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:43.647177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248803f-b1e4-45d5-a900-536943e3f5d5 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:43.664387 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e1e266-459c-45e4-a386-194750c4883d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:43.666324 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e1e266-459c-45e4-a386-194750c4883d None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:43.710465 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:43.720455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:43.766449 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248803f-b1e4-45d5-a900-536943e3f5d5 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:43.774254 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 60/118] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:29:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:29:43.775290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1e266-459c-45e4-a386-194750c4883d None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:43.807417 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5209883-75dc-46c0-b0a4-64aa6362ccfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:43.808221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5209883-75dc-46c0-b0a4-64aa6362ccfd None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:43.808513 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5209883-75dc-46c0-b0a4-64aa6362ccfd None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:43.808864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5209883-75dc-46c0-b0a4-64aa6362ccfd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:43.820487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5209883-75dc-46c0-b0a4-64aa6362ccfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:29:43.829420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-d5209883-75dc-46c0-b0a4-64aa6362ccfd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:29:43.829420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:43.829420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:43.829420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:43.829420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:43.829420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:43.829420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:43.829420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:43.829420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:43.829420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:43.829420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:43.829420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:43.829420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:29:43.829420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:29:43.829420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:29:43.829420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:29:43.830292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:29:43.830292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:29:43.830292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:29:43.830292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:29:43.830292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:43.830292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:43.830292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:43.830292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:43.830292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:43.830292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:43.830292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:43.830292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:29:43.830292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:43.830292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:43.830292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:43.830292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:43.831069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:43.831069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:43.831069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:43.831069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:43.831069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:29:43.831069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:29:43.831069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:29:43.831069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:29:43.831069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:29:43.831069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:29:43.831069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:29:43.831069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:29:43.831069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 59/119] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:29:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:29:43.831691 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5209883-75dc-46c0-b0a4-64aa6362ccfd None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:43.895445 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e1e266-459c-45e4-a386-194750c4883d None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:43.938700 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f76fd-ff25-46bb-98e1-ea5f8061a6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:43.939515 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f76fd-ff25-46bb-98e1-ea5f8061a6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:43.939811 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f76fd-ff25-46bb-98e1-ea5f8061a6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:43.940096 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f76fd-ff25-46bb-98e1-ea5f8061a6c7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:43.945634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5209883-75dc-46c0-b0a4-64aa6362ccfd None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:43.952541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f76fd-ff25-46bb-98e1-ea5f8061a6c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:29:43.959001 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:43.960673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-2e1f76fd-ff25-46bb-98e1-ea5f8061a6c7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 19:29:43.960673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:43.960673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:43.960673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:43.960673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:43.960673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:43.960673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:43.960673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:43.960673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:43.960673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:43.960673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:43.960673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:43.960673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:29:43.960673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:29:43.960673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:29:43.960673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:29:43.961745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:29:43.961745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:29:43.961745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:29:43.961745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:29:43.961745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:43.961745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:43.961745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:43.961745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:43.961745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:43.961745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:43.961745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:43.961745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:29:43.961745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:43.961745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:43.961745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:43.961745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:43.962699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:43.962699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:43.962699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:43.962699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:43.962699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:29:43.962699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:29:43.962699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:29:43.962699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:29:43.962699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:29:43.962699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:29:43.962699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 19:29:43.962699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:29:43.962699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 61/120] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:29:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:29:43.962699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f76fd-ff25-46bb-98e1-ea5f8061a6c7 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:43.983672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:43.991857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c7710-4502-4fca-811d-589c8c1e4af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:43.992726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c7710-4502-4fca-811d-589c8c1e4af7 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:43.993036 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c7710-4502-4fca-811d-589c8c1e4af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:43.993315 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6c7710-4502-4fca-811d-589c8c1e4af7 None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:43.994501 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6c7710-4502-4fca-811d-589c8c1e4af7 None admin] RBAC: Authorizing `identity:get_project(project_id=b30f38a32f6d4982bd829828ccac6fa7)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:43.995945 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6c7710-4502-4fca-811d-589c8c1e4af7 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:43.997671 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 60/121] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:29:43 2022] GET /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:43.998423 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c7710-4502-4fca-811d-589c8c1e4af7 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:44.084095 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f76fd-ff25-46bb-98e1-ea5f8061a6c7 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:44.107747 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6c7710-4502-4fca-811d-589c8c1e4af7 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:44.119998 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c376dad-4127-4b93-b7a7-263a508d0719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:44.120802 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c376dad-4127-4b93-b7a7-263a508d0719 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:44.121534 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c376dad-4127-4b93-b7a7-263a508d0719 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:44.121534 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c376dad-4127-4b93-b7a7-263a508d0719 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:44.133653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c376dad-4127-4b93-b7a7-263a508d0719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:29:44.143283 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0a0a4-3c68-4433-8ad6-cfa7a8e4ac58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:44.144154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-3c376dad-4127-4b93-b7a7-263a508d0719 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:29:44.144154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:44.144154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:44.144154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:44.144154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:44.144154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:44.144154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:44.144154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:44.144154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:44.144154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:44.144154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:44.144154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:44.144154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:29:44.144154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:29:44.144154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:29:44.144154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:29:44.145139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:29:44.145139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:29:44.145139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:29:44.145139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:29:44.145139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:44.145139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:44.145139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:44.145139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:44.145139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:44.145139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:44.145139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:44.145139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:29:44.145139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:44.145139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:44.145139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:44.145139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:44.145946 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:44.145946 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:44.145946 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:44.145946 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:44.145946 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:29:44.145946 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:29:44.145946 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:29:44.145946 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:29:44.145946 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:29:44.145946 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:29:44.145946 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:29:44.145946 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:29:44.146538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0a0a4-3c68-4433-8ad6-cfa7a8e4ac58 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:44.146538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0a0a4-3c68-4433-8ad6-cfa7a8e4ac58 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:44.146538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0a0a4-3c68-4433-8ad6-cfa7a8e4ac58 None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:44.146795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 62/122] 10.222.0.13 () {56 vars in 1114 bytes} [Fri May 27 19:29:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:29:44.146795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c376dad-4127-4b93-b7a7-263a508d0719 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:44.148224 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad0a0a4-3c68-4433-8ad6-cfa7a8e4ac58 None admin] RBAC: Authorizing `identity:get_project(project_id=b30f38a32f6d4982bd829828ccac6fa7)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:44.149727 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad0a0a4-3c68-4433-8ad6-cfa7a8e4ac58 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:44.151742 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 61/123] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:29:43 2022] GET /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:44.152450 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0a0a4-3c68-4433-8ad6-cfa7a8e4ac58 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:44.203411 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:44.229771 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:44.266086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c376dad-4127-4b93-b7a7-263a508d0719 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:44.268375 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0a0a4-3c68-4433-8ad6-cfa7a8e4ac58 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:44.306546 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7e650-2a05-423b-bf68-436dc3c66484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:44.307234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7e650-2a05-423b-bf68-436dc3c66484 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:44.307524 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7e650-2a05-423b-bf68-436dc3c66484 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:44.307809 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c7e650-2a05-423b-bf68-436dc3c66484 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:44.308570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7e650-2a05-423b-bf68-436dc3c66484 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:44.308887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7e650-2a05-423b-bf68-436dc3c66484 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:44.309995 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e4066-507d-4c15-b14a-f28846e5ff68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:44.310330 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c7e650-2a05-423b-bf68-436dc3c66484 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:44.310666 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e4066-507d-4c15-b14a-f28846e5ff68 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:44.310954 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e4066-507d-4c15-b14a-f28846e5ff68 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:44.311257 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e4066-507d-4c15-b14a-f28846e5ff68 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:44.312559 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e4066-507d-4c15-b14a-f28846e5ff68 None admin] RBAC: Adding query filter params (name=member) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:44.312872 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e4066-507d-4c15-b14a-f28846e5ff68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:44.314283 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e4066-507d-4c15-b14a-f28846e5ff68 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:44.320449 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 63/124] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:29:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:44.321169 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7e650-2a05-423b-bf68-436dc3c66484 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:44.332992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 62/125] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:29:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:44.332992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e4066-507d-4c15-b14a-f28846e5ff68 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:44.437842 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c7e650-2a05-423b-bf68-436dc3c66484 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:44.444659 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e4066-507d-4c15-b14a-f28846e5ff68 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:44.454388 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:44.469621 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:44.475036 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dae1c6-86f5-45c3-a736-23f004ec8adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:44.475940 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dae1c6-86f5-45c3-a736-23f004ec8adf None admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:44.476223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dae1c6-86f5-45c3-a736-23f004ec8adf None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:44.476504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dae1c6-86f5-45c3-a736-23f004ec8adf None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7/users/02e527c314c64f03bbc6d51e8e772a09/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:44.478731 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dae1c6-86f5-45c3-a736-23f004ec8adf None admin] RBAC: Authorizing `identity:create_grant(project_id=b30f38a32f6d4982bd829828ccac6fa7, user_id=02e527c314c64f03bbc6d51e8e772a09, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:44.480215 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dae1c6-86f5-45c3-a736-23f004ec8adf None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:44.480569 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09113bf-d29e-4767-ad1e-029e9d06b30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:44.481282 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09113bf-d29e-4767-ad1e-029e9d06b30a None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:44.481577 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09113bf-d29e-4767-ad1e-029e9d06b30a None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:44.481859 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09113bf-d29e-4767-ad1e-029e9d06b30a None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:44.482648 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09113bf-d29e-4767-ad1e-029e9d06b30a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:44.482966 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09113bf-d29e-4767-ad1e-029e9d06b30a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:44.485525 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09113bf-d29e-4767-ad1e-029e9d06b30a None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:44.494287 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 63/126] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:29:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:44.495367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09113bf-d29e-4767-ad1e-029e9d06b30a None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:44.576136 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 64/127] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:29:44 2022] PUT /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7/users/02e527c314c64f03bbc6d51e8e772a09/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:29:44.577191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dae1c6-86f5-45c3-a736-23f004ec8adf None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:44.651174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09113bf-d29e-4767-ad1e-029e9d06b30a None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:44.699677 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dae1c6-86f5-45c3-a736-23f004ec8adf None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:44.704782 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e24ac7-7d4d-4af5-9011-f6a81e089c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:44.705682 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e24ac7-7d4d-4af5-9011-f6a81e089c18 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:44.706044 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e24ac7-7d4d-4af5-9011-f6a81e089c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:44.706394 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e24ac7-7d4d-4af5-9011-f6a81e089c18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:44.707365 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e24ac7-7d4d-4af5-9011-f6a81e089c18 None admin] RBAC: Adding query filter params (role.id=9170e7d540b446f3b5507688ca335ecc, scope.project.id=b30f38a32f6d4982bd829828ccac6fa7, user.id=e00abb1711cb497187586e15b695c7d0) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:44.707755 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e24ac7-7d4d-4af5-9011-f6a81e089c18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:44.709356 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e24ac7-7d4d-4af5-9011-f6a81e089c18 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:44.718471 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 64/128] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:29:44 2022] GET /identity/v3/role_assignments?user.id=e00abb1711cb497187586e15b695c7d0&scope.project.id=b30f38a32f6d4982bd829828ccac6fa7&role.id=9170e7d540b446f3b5507688ca335ecc => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:44.719842 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e24ac7-7d4d-4af5-9011-f6a81e089c18 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:44.755900 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79177701-ea13-4ef8-ab30-3acf9e070522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:44.757184 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79177701-ea13-4ef8-ab30-3acf9e070522 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:44.757661 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79177701-ea13-4ef8-ab30-3acf9e070522 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:44.758118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79177701-ea13-4ef8-ab30-3acf9e070522 None admin] PATH_INFO: `/v3/users/736ecd08572d41d59ba0fe2db80e313a` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:44.763287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79177701-ea13-4ef8-ab30-3acf9e070522 None admin] RBAC: Authorizing `identity:get_user(user_id=736ecd08572d41d59ba0fe2db80e313a)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:44.765785 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79177701-ea13-4ef8-ab30-3acf9e070522 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:44.767887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 65/129] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:29:44 2022] GET /identity/v3/users/736ecd08572d41d59ba0fe2db80e313a => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:44.770250 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79177701-ea13-4ef8-ab30-3acf9e070522 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:44.795383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:44.871826 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:44.882335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e24ac7-7d4d-4af5-9011-f6a81e089c18 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:44.921949 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79177701-ea13-4ef8-ab30-3acf9e070522 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:44.925764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed32914-f6eb-4c44-bc3b-a970879bf2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:44.926058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed32914-f6eb-4c44-bc3b-a970879bf2bb None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:44.926360 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed32914-f6eb-4c44-bc3b-a970879bf2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:44.926645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed32914-f6eb-4c44-bc3b-a970879bf2bb None admin] PATH_INFO: `/v3/users/02e527c314c64f03bbc6d51e8e772a09` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:44.927933 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed32914-f6eb-4c44-bc3b-a970879bf2bb None admin] RBAC: Authorizing `identity:get_user(user_id=02e527c314c64f03bbc6d51e8e772a09)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:44.929477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed32914-f6eb-4c44-bc3b-a970879bf2bb None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:44.931620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 65/130] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:29:44 2022] GET /identity/v3/users/02e527c314c64f03bbc6d51e8e772a09 => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:44.932733 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed32914-f6eb-4c44-bc3b-a970879bf2bb None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:44.971098 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6e905-f868-45db-b4a5-f05bc3ed0547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:44.972286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6e905-f868-45db-b4a5-f05bc3ed0547 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:44.972836 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6e905-f868-45db-b4a5-f05bc3ed0547 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:44.973314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6e905-f868-45db-b4a5-f05bc3ed0547 None admin] PATH_INFO: `/v3/users/02e527c314c64f03bbc6d51e8e772a09` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:44.974770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e6e905-f868-45db-b4a5-f05bc3ed0547 None admin] RBAC: Authorizing `identity:get_user(user_id=02e527c314c64f03bbc6d51e8e772a09)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:44.976435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e6e905-f868-45db-b4a5-f05bc3ed0547 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:44.978292 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 66/131] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:29:44 2022] GET /identity/v3/users/02e527c314c64f03bbc6d51e8e772a09 => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:44.979310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6e905-f868-45db-b4a5-f05bc3ed0547 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:45.085798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6e905-f868-45db-b4a5-f05bc3ed0547 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:45.117452 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:45.122685 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00538c1-4fdb-41e3-8eac-87808641a327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:45.123850 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00538c1-4fdb-41e3-8eac-87808641a327 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:45.124326 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00538c1-4fdb-41e3-8eac-87808641a327 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:45.124945 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00538c1-4fdb-41e3-8eac-87808641a327 None admin] PATH_INFO: `/v3/projects/c2c64289866e4b5d949e9a8c4012f2c8` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:45.126538 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00538c1-4fdb-41e3-8eac-87808641a327 None admin] RBAC: Authorizing `identity:get_project(project_id=c2c64289866e4b5d949e9a8c4012f2c8)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:45.133317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00538c1-4fdb-41e3-8eac-87808641a327 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:45.133317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 67/132] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:29:44 2022] GET /identity/v3/projects/c2c64289866e4b5d949e9a8c4012f2c8 => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:45.133317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00538c1-4fdb-41e3-8eac-87808641a327 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:45.140676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed32914-f6eb-4c44-bc3b-a970879bf2bb None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:45.142515 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:45.236992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16112a-aeef-4d50-8129-24520a77fa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:45.237877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16112a-aeef-4d50-8129-24520a77fa7f None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:45.238174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16112a-aeef-4d50-8129-24520a77fa7f None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:45.238463 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c16112a-aeef-4d50-8129-24520a77fa7f None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:45.239057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00538c1-4fdb-41e3-8eac-87808641a327 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:45.239696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16112a-aeef-4d50-8129-24520a77fa7f None admin] RBAC: Authorizing `identity:get_project(project_id=b30f38a32f6d4982bd829828ccac6fa7)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:45.241197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c16112a-aeef-4d50-8129-24520a77fa7f None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:45.243141 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 66/133] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:29:44 2022] GET /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7 => generated 298 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:45.248331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16112a-aeef-4d50-8129-24520a77fa7f None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:45.282810 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98b23e-c4c0-4fea-87de-399f6157fdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:45.284086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98b23e-c4c0-4fea-87de-399f6157fdbe None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:45.284551 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98b23e-c4c0-4fea-87de-399f6157fdbe None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:45.285035 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98b23e-c4c0-4fea-87de-399f6157fdbe None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:45.286543 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de98b23e-c4c0-4fea-87de-399f6157fdbe None admin] RBAC: Authorizing `identity:get_project(project_id=b30f38a32f6d4982bd829828ccac6fa7)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:45.288218 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de98b23e-c4c0-4fea-87de-399f6157fdbe None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:45.290254 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 68/134] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:29:45 2022] GET /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:45.291166 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98b23e-c4c0-4fea-87de-399f6157fdbe None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:45.393041 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:45.430159 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c16112a-aeef-4d50-8129-24520a77fa7f None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:45.473606 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de98b23e-c4c0-4fea-87de-399f6157fdbe None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:45.475521 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26240231-6d88-4295-858f-4f8c9f578ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:45.477548 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26240231-6d88-4295-858f-4f8c9f578ff2 None admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:45.478022 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26240231-6d88-4295-858f-4f8c9f578ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:45.478480 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26240231-6d88-4295-858f-4f8c9f578ff2 None admin] PATH_INFO: `/v3/projects/c2c64289866e4b5d949e9a8c4012f2c8/users/02e527c314c64f03bbc6d51e8e772a09/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:45.480940 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26240231-6d88-4295-858f-4f8c9f578ff2 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2c64289866e4b5d949e9a8c4012f2c8, user_id=02e527c314c64f03bbc6d51e8e772a09, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:45.483196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26240231-6d88-4295-858f-4f8c9f578ff2 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:45.522778 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2814f18-9ebb-45ff-a671-3b85e81c9172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:45.522778 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2814f18-9ebb-45ff-a671-3b85e81c9172 None admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:45.522778 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2814f18-9ebb-45ff-a671-3b85e81c9172 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:45.522778 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2814f18-9ebb-45ff-a671-3b85e81c9172 None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7/users/736ecd08572d41d59ba0fe2db80e313a/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:45.524650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2814f18-9ebb-45ff-a671-3b85e81c9172 None admin] RBAC: Authorizing `identity:create_grant(project_id=b30f38a32f6d4982bd829828ccac6fa7, user_id=736ecd08572d41d59ba0fe2db80e313a, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:45.526266 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2814f18-9ebb-45ff-a671-3b85e81c9172 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:45.591137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 67/135] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:29:45 2022] PUT /identity/v3/projects/c2c64289866e4b5d949e9a8c4012f2c8/users/02e527c314c64f03bbc6d51e8e772a09/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:29:45.592616 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26240231-6d88-4295-858f-4f8c9f578ff2 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:45.620711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 69/136] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:29:45 2022] PUT /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7/users/736ecd08572d41d59ba0fe2db80e313a/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:29:45.623047 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c12ee-9e34-40bc-8c4d-66e2733b26ce None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:45.628165 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c12ee-9e34-40bc-8c4d-66e2733b26ce None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:45.628165 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c12ee-9e34-40bc-8c4d-66e2733b26ce None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:45.628165 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 70/137] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:45.634749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e943bd-1900-4b17-bacb-a78ce01ce399 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:45.635258 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e943bd-1900-4b17-bacb-a78ce01ce399 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:45.635825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e943bd-1900-4b17-bacb-a78ce01ce399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:45.662447 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-f5e943bd-1900-4b17-bacb-a78ce01ce399 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:45.662447 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-f5e943bd-1900-4b17-bacb-a78ce01ce399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:45.666344 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:45.765898 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 71/138] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 19:29:45.787921 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e943bd-1900-4b17-bacb-a78ce01ce399 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:45.826333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26240231-6d88-4295-858f-4f8c9f578ff2 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:45.881348 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:45.923300 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e943bd-1900-4b17-bacb-a78ce01ce399 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:45.934416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc6e81-05ae-410e-a327-37a3cf7af20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:45.935345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc6e81-05ae-410e-a327-37a3cf7af20c None admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:45.935642 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc6e81-05ae-410e-a327-37a3cf7af20c None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:45.935926 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc6e81-05ae-410e-a327-37a3cf7af20c None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7/users/02e527c314c64f03bbc6d51e8e772a09/roles/eda4ac05bc364224a7f319e7ded54073` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:45.939968 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc6e81-05ae-410e-a327-37a3cf7af20c None admin] RBAC: Authorizing `identity:create_grant(project_id=b30f38a32f6d4982bd829828ccac6fa7, user_id=02e527c314c64f03bbc6d51e8e772a09, role_id=eda4ac05bc364224a7f319e7ded54073)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:45.941937 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc6e81-05ae-410e-a327-37a3cf7af20c None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:45.970232 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c185e3-499b-46a6-90b8-8c7d9d9d5f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:45.971055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c185e3-499b-46a6-90b8-8c7d9d9d5f82 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:45.971341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c185e3-499b-46a6-90b8-8c7d9d9d5f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:45.971676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c185e3-499b-46a6-90b8-8c7d9d9d5f82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:45.972916 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c185e3-499b-46a6-90b8-8c7d9d9d5f82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:45.974555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c185e3-499b-46a6-90b8-8c7d9d9d5f82 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:45.976233 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 72/139] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:29:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:29:45.982012 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c185e3-499b-46a6-90b8-8c7d9d9d5f82 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:46.020610 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 68/140] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:29:45 2022] PUT /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7/users/02e527c314c64f03bbc6d51e8e772a09/roles/eda4ac05bc364224a7f319e7ded54073 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:29:46.118597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c185e3-499b-46a6-90b8-8c7d9d9d5f82 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:46.153354 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:46.164109 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08342336-e7fe-42bf-b9e9-292e61c0b53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:46.164807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08342336-e7fe-42bf-b9e9-292e61c0b53e None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:46.165125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08342336-e7fe-42bf-b9e9-292e61c0b53e None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:46.165407 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08342336-e7fe-42bf-b9e9-292e61c0b53e None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:46.166156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08342336-e7fe-42bf-b9e9-292e61c0b53e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:46.167633 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08342336-e7fe-42bf-b9e9-292e61c0b53e None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:46.272790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 73/141] 10.222.0.13 () {60 vars in 1132 bytes} [Fri May 27 19:29:45 2022] POST /identity/v3/users => generated 283 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:29:47.513736 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3a86e-8444-494c-a8b0-f040dad5301d None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:47.514794 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3a86e-8444-494c-a8b0-f040dad5301d None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:47.515119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3a86e-8444-494c-a8b0-f040dad5301d None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:47.516749 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 69/142] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:47.522683 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cfb39-e9f4-4b18-95da-ad1df4e627cf None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:47.522683 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cfb39-e9f4-4b18-95da-ad1df4e627cf None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:47.522683 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062cfb39-e9f4-4b18-95da-ad1df4e627cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:47.541982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-062cfb39-e9f4-4b18-95da-ad1df4e627cf None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:47.543175 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-062cfb39-e9f4-4b18-95da-ad1df4e627cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:47.636758 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 74/143] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:47.667948 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3a86e-8444-494c-a8b0-f040dad5301d None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:47.707323 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334813a-4306-46b1-87bc-51c03ac3dc2c None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:47.707639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334813a-4306-46b1-87bc-51c03ac3dc2c None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:47.707893 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5334813a-4306-46b1-87bc-51c03ac3dc2c None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:47.709616 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 75/144] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:47.715072 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c3131-43cd-4ab5-8ff4-d7c88dee896f None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:47.715363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c3131-43cd-4ab5-8ff4-d7c88dee896f None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:47.715647 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c3131-43cd-4ab5-8ff4-d7c88dee896f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:47.750432 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-0c3c3131-43cd-4ab5-8ff4-d7c88dee896f None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:47.751882 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-0c3c3131-43cd-4ab5-8ff4-d7c88dee896f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:47.818900 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:47.843526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea3a86e-8444-494c-a8b0-f040dad5301d None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:47.856789 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 76/145] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 19:29:47.916552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6eaa7-d93d-4cad-a6a9-581e24d1a583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:47.917411 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6eaa7-d93d-4cad-a6a9-581e24d1a583 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:47.917701 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6eaa7-d93d-4cad-a6a9-581e24d1a583 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:47.917983 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6eaa7-d93d-4cad-a6a9-581e24d1a583 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:47.921773 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c3131-43cd-4ab5-8ff4-d7c88dee896f None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:47.945409 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6eaa7-d93d-4cad-a6a9-581e24d1a583 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:29:47.953776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-90d6eaa7-d93d-4cad-a6a9-581e24d1a583 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 19:29:47.953776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:47.953776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:47.953776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:47.953776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:47.953776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:47.953776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:47.953776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:47.953776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:47.953776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:47.953776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:47.953776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:47.953776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:29:47.953776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:29:47.953776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:29:47.953776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:29:47.954695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:29:47.954695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:29:47.954695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:29:47.954695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:29:47.954695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:47.954695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:47.954695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:47.954695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:47.954695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:47.954695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:47.954695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:47.954695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:29:47.954695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:47.954695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:47.954695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:47.954695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:47.956378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:47.956378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:47.956378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:47.956378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:47.956378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:29:47.956378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:29:47.956378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:29:47.956378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:29:47.956378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:29:47.956378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:29:47.956378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 19:29:47.956378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:29:47.956378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 70/146] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:29:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:29:47.960884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6eaa7-d93d-4cad-a6a9-581e24d1a583 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:47.992913 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:48.070431 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c3131-43cd-4ab5-8ff4-d7c88dee896f None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:48.133725 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb135f-2088-4487-b209-1bfd901546ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:48.136973 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb135f-2088-4487-b209-1bfd901546ad None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:48.136973 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb135f-2088-4487-b209-1bfd901546ad None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:48.136973 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb135f-2088-4487-b209-1bfd901546ad None admin] PATH_INFO: `/v3/roles/member` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:48.164233 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fb135f-2088-4487-b209-1bfd901546ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:29:48.181597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-c6fb135f-2088-4487-b209-1bfd901546ad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 19:29:48.181597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:48.181597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:48.181597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:48.181597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:48.181597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:48.181597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:48.181597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:48.181597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:48.181597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:48.181597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:48.181597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:48.181597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:29:48.181597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:29:48.181597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:29:48.181597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:29:48.182514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:29:48.182514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:29:48.182514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:29:48.182514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:29:48.182514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:48.182514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:48.182514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:48.182514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:48.182514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:48.182514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:48.182514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:48.182514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:29:48.182514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:48.182514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:48.182514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:48.182514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:48.183290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:48.183290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:48.183290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:48.183290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:48.183290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:29:48.183290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:29:48.183290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:29:48.183290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:29:48.183290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:29:48.183290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:29:48.183290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 19:29:48.183290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:29:48.183290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 77/147] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:29:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:29:48.192146 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb135f-2088-4487-b209-1bfd901546ad None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:48.194924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6eaa7-d93d-4cad-a6a9-581e24d1a583 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:48.245748 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c00c68-dfb9-47cc-a096-0f90ffb1d681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:48.246846 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c00c68-dfb9-47cc-a096-0f90ffb1d681 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:48.247347 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c00c68-dfb9-47cc-a096-0f90ffb1d681 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:48.247820 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c00c68-dfb9-47cc-a096-0f90ffb1d681 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:48.249403 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c00c68-dfb9-47cc-a096-0f90ffb1d681 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:48.250004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c00c68-dfb9-47cc-a096-0f90ffb1d681 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:48.251607 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c00c68-dfb9-47cc-a096-0f90ffb1d681 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:48.260723 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 71/148] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:29:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:48.266469 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c00c68-dfb9-47cc-a096-0f90ffb1d681 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:48.295651 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:48.428143 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:48.439381 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb135f-2088-4487-b209-1bfd901546ad None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:48.441906 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c00c68-dfb9-47cc-a096-0f90ffb1d681 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:48.507032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971b6b0-05bc-4164-b82d-c66e886a816d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:48.507228 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd27281-3bbc-4cc5-8c8f-fa86340b739d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:48.507765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd27281-3bbc-4cc5-8c8f-fa86340b739d None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:48.508054 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd27281-3bbc-4cc5-8c8f-fa86340b739d None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:48.508339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd27281-3bbc-4cc5-8c8f-fa86340b739d None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:48.508681 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971b6b0-05bc-4164-b82d-c66e886a816d None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:48.509139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd27281-3bbc-4cc5-8c8f-fa86340b739d None admin] RBAC: Adding query filter params (name=member) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:48.509246 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971b6b0-05bc-4164-b82d-c66e886a816d None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:48.509460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd27281-3bbc-4cc5-8c8f-fa86340b739d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:48.510087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971b6b0-05bc-4164-b82d-c66e886a816d None admin] PATH_INFO: `/v3/users/e00abb1711cb497187586e15b695c7d0` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:48.512864 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd27281-3bbc-4cc5-8c8f-fa86340b739d None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:48.513817 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b971b6b0-05bc-4164-b82d-c66e886a816d None admin] RBAC: Authorizing `identity:get_user(user_id=e00abb1711cb497187586e15b695c7d0)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:48.516264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b971b6b0-05bc-4164-b82d-c66e886a816d None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:48.518259 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 72/149] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:29:48 2022] GET /identity/v3/users/e00abb1711cb497187586e15b695c7d0 => generated 289 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:48.523789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971b6b0-05bc-4164-b82d-c66e886a816d None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:48.526651 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 78/150] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:29:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:29:48.535959 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd27281-3bbc-4cc5-8c8f-fa86340b739d None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:48.650184 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd27281-3bbc-4cc5-8c8f-fa86340b739d None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:48.674172 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:48.692223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee25fda-edc4-4b25-bdb9-75b3bfb981b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:48.693133 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee25fda-edc4-4b25-bdb9-75b3bfb981b2 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:48.693447 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee25fda-edc4-4b25-bdb9-75b3bfb981b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:48.693733 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee25fda-edc4-4b25-bdb9-75b3bfb981b2 None admin] PATH_INFO: `/v3/users/02e527c314c64f03bbc6d51e8e772a09` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:48.697721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee25fda-edc4-4b25-bdb9-75b3bfb981b2 None admin] RBAC: Authorizing `identity:get_user(user_id=02e527c314c64f03bbc6d51e8e772a09)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:48.699316 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee25fda-edc4-4b25-bdb9-75b3bfb981b2 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:48.701217 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 79/151] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:29:48 2022] GET /identity/v3/users/02e527c314c64f03bbc6d51e8e772a09 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:29:48.713913 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971b6b0-05bc-4164-b82d-c66e886a816d None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:48.718573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee25fda-edc4-4b25-bdb9-75b3bfb981b2 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:48.756744 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:48.774628 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad11876-07e7-4448-ac47-767fa3f5fad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:48.775785 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad11876-07e7-4448-ac47-767fa3f5fad3 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:48.776400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad11876-07e7-4448-ac47-767fa3f5fad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:48.776899 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad11876-07e7-4448-ac47-767fa3f5fad3 None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:48.782271 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad11876-07e7-4448-ac47-767fa3f5fad3 None admin] RBAC: Authorizing `identity:get_project(project_id=b30f38a32f6d4982bd829828ccac6fa7)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:48.783952 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad11876-07e7-4448-ac47-767fa3f5fad3 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:48.786180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 73/152] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:29:48 2022] GET /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:29:48.792536 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad11876-07e7-4448-ac47-767fa3f5fad3 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:48.856263 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee25fda-edc4-4b25-bdb9-75b3bfb981b2 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:48.898176 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89914d7b-3ad0-40fa-bbd0-9a2f48f55048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:48.898844 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89914d7b-3ad0-40fa-bbd0-9a2f48f55048 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:48.899149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89914d7b-3ad0-40fa-bbd0-9a2f48f55048 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:48.899433 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89914d7b-3ad0-40fa-bbd0-9a2f48f55048 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:49.004618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad11876-07e7-4448-ac47-767fa3f5fad3 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:49.079495 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89914d7b-3ad0-40fa-bbd0-9a2f48f55048 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:49.079874 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f90343-e229-4622-8b2c-d412d90c309d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:49.080807 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f90343-e229-4622-8b2c-d412d90c309d None admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:49.081032 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89914d7b-3ad0-40fa-bbd0-9a2f48f55048 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:49.081144 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f90343-e229-4622-8b2c-d412d90c309d None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:49.081397 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f90343-e229-4622-8b2c-d412d90c309d None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7/users/e00abb1711cb497187586e15b695c7d0/roles/9170e7d540b446f3b5507688ca335ecc` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:49.083856 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f90343-e229-4622-8b2c-d412d90c309d None admin] RBAC: Authorizing `identity:create_grant(project_id=b30f38a32f6d4982bd829828ccac6fa7, user_id=e00abb1711cb497187586e15b695c7d0, role_id=9170e7d540b446f3b5507688ca335ecc)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:49.085398 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f90343-e229-4622-8b2c-d412d90c309d None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:49.137967 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:49.195245 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 74/153] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:29:48 2022] PUT /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7/users/e00abb1711cb497187586e15b695c7d0/roles/9170e7d540b446f3b5507688ca335ecc => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:29:49.226878 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 80/154] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:29:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 19:29:49.229437 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e40b5-19a4-49d8-aff5-5a7e0ded16a1 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:49.229754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e40b5-19a4-49d8-aff5-5a7e0ded16a1 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:49.230044 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e40b5-19a4-49d8-aff5-5a7e0ded16a1 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:49.231438 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 75/155] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:49.233843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89914d7b-3ad0-40fa-bbd0-9a2f48f55048 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:49.247964 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00b635-2c76-457e-89be-554ca71b4e52 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:49.249142 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00b635-2c76-457e-89be-554ca71b4e52 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:49.249453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac00b635-2c76-457e-89be-554ca71b4e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:49.279941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-ac00b635-2c76-457e-89be-554ca71b4e52 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:49.281490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-ac00b635-2c76-457e-89be-554ca71b4e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:49.283882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:49.463474 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 76/156] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:49.480938 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89914d7b-3ad0-40fa-bbd0-9a2f48f55048 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:49.534579 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac00b635-2c76-457e-89be-554ca71b4e52 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:49.551065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:49.566743 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8a2c6-76ac-407e-96e1-2dda082f9b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:49.567632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8a2c6-76ac-407e-96e1-2dda082f9b96 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:49.567913 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8a2c6-76ac-407e-96e1-2dda082f9b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:49.570975 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8a2c6-76ac-407e-96e1-2dda082f9b96 None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:49.570975 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8a2c6-76ac-407e-96e1-2dda082f9b96 None admin] RBAC: Authorizing `identity:get_project(project_id=b30f38a32f6d4982bd829828ccac6fa7)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:49.570975 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea8a2c6-76ac-407e-96e1-2dda082f9b96 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:49.572625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 81/157] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:29:49 2022] GET /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7 => generated 298 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:49.580661 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8a2c6-76ac-407e-96e1-2dda082f9b96 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:49.789156 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac00b635-2c76-457e-89be-554ca71b4e52 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:49.816606 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8a2c6-76ac-407e-96e1-2dda082f9b96 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:49.830712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed3a6c-f8f4-4dce-be07-b49574c6bad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:49.831889 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed3a6c-f8f4-4dce-be07-b49574c6bad1 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:49.832169 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed3a6c-f8f4-4dce-be07-b49574c6bad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:49.832453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ed3a6c-f8f4-4dce-be07-b49574c6bad1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:49.840994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:49.846641 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ed3a6c-f8f4-4dce-be07-b49574c6bad1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:29:49.854408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-b6ed3a6c-f8f4-4dce-be07-b49574c6bad1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 19:29:49.854408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:49.854408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:49.854408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:49.854408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:49.854408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:49.854408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:49.854408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:49.854408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:49.854408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:49.854408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:49.854408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:49.854408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:29:49.854408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:29:49.854408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:29:49.854408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:29:49.857620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:29:49.857620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:29:49.857620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:29:49.857620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:29:49.857620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:49.857620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:49.857620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:49.857620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:49.857620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:49.857620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:49.857620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:49.857620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:29:49.857620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:49.857620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:49.857620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:49.857620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:49.858951 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:49.858951 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:49.858951 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:49.858951 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:49.858951 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:29:49.858951 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:29:49.858951 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:29:49.858951 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:29:49.858951 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:29:49.858951 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:29:49.858951 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 19:29:49.858951 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:29:49.866992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 77/158] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:29:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:29:49.867943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed3a6c-f8f4-4dce-be07-b49574c6bad1 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:49.890582 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd475c-cfcd-4e02-b4e5-4a96de93c397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:49.891251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd475c-cfcd-4e02-b4e5-4a96de93c397 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:49.891535 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd475c-cfcd-4e02-b4e5-4a96de93c397 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:49.891815 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd475c-cfcd-4e02-b4e5-4a96de93c397 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:49.892660 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd475c-cfcd-4e02-b4e5-4a96de93c397 None admin] RBAC: Adding query filter params (role.id=cb43a575e47147c799eec2f362f4fe0e, scope.project.id=b30f38a32f6d4982bd829828ccac6fa7, user.id=02e527c314c64f03bbc6d51e8e772a09) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:49.892994 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd475c-cfcd-4e02-b4e5-4a96de93c397 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:49.894454 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd475c-cfcd-4e02-b4e5-4a96de93c397 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:49.907002 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 82/159] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:29:49 2022] GET /identity/v3/role_assignments?user.id=02e527c314c64f03bbc6d51e8e772a09&scope.project.id=b30f38a32f6d4982bd829828ccac6fa7&role.id=cb43a575e47147c799eec2f362f4fe0e => generated 608 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:29:50.013238 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ed3a6c-f8f4-4dce-be07-b49574c6bad1 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:50.059114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78f6d3-04fb-429a-8156-57a71b288725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:50.059716 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78f6d3-04fb-429a-8156-57a71b288725 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:50.060003 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78f6d3-04fb-429a-8156-57a71b288725 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:50.060286 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78f6d3-04fb-429a-8156-57a71b288725 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:50.061165 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78f6d3-04fb-429a-8156-57a71b288725 None admin] RBAC: Adding query filter params (name=member) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:50.061471 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78f6d3-04fb-429a-8156-57a71b288725 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:50.062913 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78f6d3-04fb-429a-8156-57a71b288725 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:50.077188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 78/160] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:29:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:50.082902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd475c-cfcd-4e02-b4e5-4a96de93c397 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:50.186101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:50.245382 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd475c-cfcd-4e02-b4e5-4a96de93c397 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:50.292450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011f8ed-843e-48ff-9fe4-8ef2511f7d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:50.293375 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011f8ed-843e-48ff-9fe4-8ef2511f7d0d None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:50.293663 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011f8ed-843e-48ff-9fe4-8ef2511f7d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:50.293944 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011f8ed-843e-48ff-9fe4-8ef2511f7d0d None admin] PATH_INFO: `/v3/users/02e527c314c64f03bbc6d51e8e772a09` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:50.295780 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011f8ed-843e-48ff-9fe4-8ef2511f7d0d None admin] RBAC: Authorizing `identity:get_user(user_id=02e527c314c64f03bbc6d51e8e772a09)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:50.297296 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a011f8ed-843e-48ff-9fe4-8ef2511f7d0d None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:50.299200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 83/161] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:29:50 2022] GET /identity/v3/users/02e527c314c64f03bbc6d51e8e772a09 => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:50.304587 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78f6d3-04fb-429a-8156-57a71b288725 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:50.376988 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:50.435026 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78f6d3-04fb-429a-8156-57a71b288725 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:50.493751 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74829af4-c631-4504-b521-ad6438ee3db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:50.494409 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74829af4-c631-4504-b521-ad6438ee3db9 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:50.494694 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74829af4-c631-4504-b521-ad6438ee3db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:50.494974 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74829af4-c631-4504-b521-ad6438ee3db9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:50.597343 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74829af4-c631-4504-b521-ad6438ee3db9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:50.598861 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74829af4-c631-4504-b521-ad6438ee3db9 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:50.615378 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc21a1-542d-45cc-ad24-36476e7db4d0 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:50.615877 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc21a1-542d-45cc-ad24-36476e7db4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:50.615950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc21a1-542d-45cc-ad24-36476e7db4d0 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:50.617332 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 84/162] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 19:29:50.623030 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51927b83-be54-4ccc-9a82-34af4e4153f8 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:50.624072 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51927b83-be54-4ccc-9a82-34af4e4153f8 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:50.624583 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51927b83-be54-4ccc-9a82-34af4e4153f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:50.664008 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-51927b83-be54-4ccc-9a82-34af4e4153f8 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:50.665149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-51927b83-be54-4ccc-9a82-34af4e4153f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:50.741826 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:50.805759 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 85/163] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:50.835313 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 79/164] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:29:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:29:50.841265 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51927b83-be54-4ccc-9a82-34af4e4153f8 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:50.846491 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74829af4-c631-4504-b521-ad6438ee3db9 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:50.965334 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:51.019108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51927b83-be54-4ccc-9a82-34af4e4153f8 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:51.083627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74829af4-c631-4504-b521-ad6438ee3db9 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:51.097731 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb144f5-dc48-4bb7-98ab-d1487a908a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:51.098630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb144f5-dc48-4bb7-98ab-d1487a908a8e None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:51.098918 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb144f5-dc48-4bb7-98ab-d1487a908a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:51.099197 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb144f5-dc48-4bb7-98ab-d1487a908a8e None admin] PATH_INFO: `/v3/projects/c2c64289866e4b5d949e9a8c4012f2c8` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:51.100486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb144f5-dc48-4bb7-98ab-d1487a908a8e None admin] RBAC: Authorizing `identity:get_project(project_id=c2c64289866e4b5d949e9a8c4012f2c8)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:51.102031 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb144f5-dc48-4bb7-98ab-d1487a908a8e None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:51.103780 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 86/165] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:29:50 2022] GET /identity/v3/projects/c2c64289866e4b5d949e9a8c4012f2c8 => generated 312 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:51.105053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701810a-fd6a-4e37-b7d7-f1b7df9d895e None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:51.105343 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701810a-fd6a-4e37-b7d7-f1b7df9d895e None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:51.105626 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701810a-fd6a-4e37-b7d7-f1b7df9d895e None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:51.106892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 87/166] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:51.130284 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701810a-fd6a-4e37-b7d7-f1b7df9d895e None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:51.140092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3715c7ec-c144-42b2-96ab-f8ddbd0cb5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:51.141033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715c7ec-c144-42b2-96ab-f8ddbd0cb5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:51.141316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715c7ec-c144-42b2-96ab-f8ddbd0cb5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:51.141589 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715c7ec-c144-42b2-96ab-f8ddbd0cb5d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:51.162535 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3715c7ec-c144-42b2-96ab-f8ddbd0cb5d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:29:51.178633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-3715c7ec-c144-42b2-96ab-f8ddbd0cb5d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:29:51.178633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:51.178633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:51.178633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:51.178633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:51.178633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:51.178633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:51.178633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:51.178633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:51.178633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:51.178633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:51.178633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:51.178633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:29:51.178633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:29:51.178633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:29:51.178633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:29:51.179585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:29:51.179585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:29:51.179585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:29:51.179585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:29:51.179585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:51.179585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:51.179585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:51.179585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:51.179585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:51.179585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:51.179585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:51.179585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:29:51.179585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:51.179585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:51.179585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:51.179585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:51.180656 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:51.180656 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:51.180656 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:51.180656 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:51.180656 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:29:51.180656 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:29:51.180656 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:29:51.180656 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:29:51.180656 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:29:51.180656 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:29:51.180656 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:29:51.180656 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:29:51.180656 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 80/167] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:29:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:29:51.182279 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08acff4-52ec-487c-a114-7b4948abb218 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:51.182279 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08acff4-52ec-487c-a114-7b4948abb218 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:51.182279 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08acff4-52ec-487c-a114-7b4948abb218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:51.213205 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-a08acff4-52ec-487c-a114-7b4948abb218 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:51.214405 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-a08acff4-52ec-487c-a114-7b4948abb218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:51.238923 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:51.314697 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701810a-fd6a-4e37-b7d7-f1b7df9d895e None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:51.334325 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 81/168] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:51.335278 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08acff4-52ec-487c-a114-7b4948abb218 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:51.392848 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:51.419990 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ce199-f71a-4289-8208-3f9d776a85d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:51.421680 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ce199-f71a-4289-8208-3f9d776a85d2 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:51.422060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ce199-f71a-4289-8208-3f9d776a85d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:51.422427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ce199-f71a-4289-8208-3f9d776a85d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:51.423869 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ce199-f71a-4289-8208-3f9d776a85d2 None admin] RBAC: Adding query filter params (role.id=cb43a575e47147c799eec2f362f4fe0e, scope.project.id=c2c64289866e4b5d949e9a8c4012f2c8, user.id=02e527c314c64f03bbc6d51e8e772a09) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:51.424204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ce199-f71a-4289-8208-3f9d776a85d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:51.426152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ce199-f71a-4289-8208-3f9d776a85d2 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:51.447604 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 88/169] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:29:51 2022] GET /identity/v3/role_assignments?user.id=02e527c314c64f03bbc6d51e8e772a09&scope.project.id=c2c64289866e4b5d949e9a8c4012f2c8&role.id=cb43a575e47147c799eec2f362f4fe0e => generated 608 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:51.448243 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ce199-f71a-4289-8208-3f9d776a85d2 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:51.579303 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08acff4-52ec-487c-a114-7b4948abb218 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:51.690172 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b2f4f7-8024-46cc-85b7-8658342352cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:51.691294 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2f4f7-8024-46cc-85b7-8658342352cb None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:51.691762 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2f4f7-8024-46cc-85b7-8658342352cb None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:51.692213 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2f4f7-8024-46cc-85b7-8658342352cb None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:51.693586 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b2f4f7-8024-46cc-85b7-8658342352cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:51.694089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b2f4f7-8024-46cc-85b7-8658342352cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:51.696429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b2f4f7-8024-46cc-85b7-8658342352cb None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:51.719742 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 82/170] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:29:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:51.722137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081f47b-9b77-43ae-be60-c8ffc68d7e17 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:51.722435 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081f47b-9b77-43ae-be60-c8ffc68d7e17 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:51.722715 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5081f47b-9b77-43ae-be60-c8ffc68d7e17 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:51.726904 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ce199-f71a-4289-8208-3f9d776a85d2 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:51.728757 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 83/171] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:51.746943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73cc48-9cf3-42ad-9119-5d87426b238c None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:51.747354 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73cc48-9cf3-42ad-9119-5d87426b238c None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:51.747534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c73cc48-9cf3-42ad-9119-5d87426b238c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:51.788422 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-5c73cc48-9cf3-42ad-9119-5d87426b238c None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:51.789576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-5c73cc48-9cf3-42ad-9119-5d87426b238c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:51.817513 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:51.849818 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f0704-bd9b-463c-8d8f-1eed98225d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:51.850705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f0704-bd9b-463c-8d8f-1eed98225d38 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:51.850996 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f0704-bd9b-463c-8d8f-1eed98225d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:51.851332 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f0704-bd9b-463c-8d8f-1eed98225d38 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:51.878008 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23f0704-bd9b-463c-8d8f-1eed98225d38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:29:51.886562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-e23f0704-bd9b-463c-8d8f-1eed98225d38 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:29:51.886562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:51.886562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:51.886562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:51.886562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:51.886562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:51.886562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:51.886562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:51.886562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:51.886562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:51.886562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:51.886562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:51.886562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:29:51.886562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:29:51.886562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:29:51.886562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:29:51.888210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:29:51.888210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:29:51.888210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:29:51.888210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:29:51.888210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:51.888210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:51.888210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:51.888210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:51.888210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:51.888210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:51.888210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:51.888210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:29:51.888210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:51.888210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:51.888210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:51.888210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:51.890211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:51.890211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:51.890211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:51.890211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:51.890211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:29:51.890211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:29:51.890211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:29:51.890211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:29:51.890211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:29:51.890211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:29:51.890211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:29:51.890211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:29:51.894555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 89/172] 10.222.0.13 () {56 vars in 1114 bytes} [Fri May 27 19:29:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 447 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:29:51.895501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f0704-bd9b-463c-8d8f-1eed98225d38 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:52.001661 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 84/173] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:52.002730 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73cc48-9cf3-42ad-9119-5d87426b238c None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:52.058752 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:52.177968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f0704-bd9b-463c-8d8f-1eed98225d38 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:52.208983 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c73cc48-9cf3-42ad-9119-5d87426b238c None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:52.224706 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b00ec0-fb3b-48fb-81a2-854a4d1dffb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:52.225850 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b00ec0-fb3b-48fb-81a2-854a4d1dffb8 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:52.226256 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b00ec0-fb3b-48fb-81a2-854a4d1dffb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:52.226536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b00ec0-fb3b-48fb-81a2-854a4d1dffb8 None admin] PATH_INFO: `/v3/users/736ecd08572d41d59ba0fe2db80e313a` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:52.227954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b00ec0-fb3b-48fb-81a2-854a4d1dffb8 None admin] RBAC: Authorizing `identity:get_user(user_id=736ecd08572d41d59ba0fe2db80e313a)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:52.229468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b00ec0-fb3b-48fb-81a2-854a4d1dffb8 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:52.231485 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 90/174] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:29:51 2022] GET /identity/v3/users/736ecd08572d41d59ba0fe2db80e313a => generated 247 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:52.232168 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b00ec0-fb3b-48fb-81a2-854a4d1dffb8 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:52.259573 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c6a0b-1122-446e-a627-be2bcd38b98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:52.260267 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c6a0b-1122-446e-a627-be2bcd38b98d None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:52.260663 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c6a0b-1122-446e-a627-be2bcd38b98d None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:52.260944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c6a0b-1122-446e-a627-be2bcd38b98d None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:52.261863 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c6a0b-1122-446e-a627-be2bcd38b98d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:52.262182 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c6a0b-1122-446e-a627-be2bcd38b98d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:52.264453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c6a0b-1122-446e-a627-be2bcd38b98d None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:52.273839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 85/175] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:29:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:52.275605 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8a7f1-26e1-4ee5-9f5a-e1e7f96026b6 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:52.275605 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8a7f1-26e1-4ee5-9f5a-e1e7f96026b6 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:52.275943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb8a7f1-26e1-4ee5-9f5a-e1e7f96026b6 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:52.277066 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 86/176] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:52.282941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb8a7f1-26e1-4ee5-9f5a-e1e7f96026b6 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:52.293169 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:52.406279 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b00ec0-fb3b-48fb-81a2-854a4d1dffb8 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:52.425722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:52.429255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb8a7f1-26e1-4ee5-9f5a-e1e7f96026b6 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:52.444719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94253ac-ff5e-4c23-9f71-f46c8583a7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:52.445660 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94253ac-ff5e-4c23-9f71-f46c8583a7cb None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:52.445956 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94253ac-ff5e-4c23-9f71-f46c8583a7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:52.446272 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94253ac-ff5e-4c23-9f71-f46c8583a7cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:52.459717 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94253ac-ff5e-4c23-9f71-f46c8583a7cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:29:52.468442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-e94253ac-ff5e-4c23-9f71-f46c8583a7cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:29:52.468442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:52.468442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:52.468442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:52.468442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:52.468442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:52.468442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:52.468442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:52.468442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:52.468442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:52.468442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:52.468442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:52.468442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:29:52.468442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:29:52.468442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:29:52.468442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:29:52.469361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:29:52.469361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:29:52.469361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:29:52.469361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:29:52.469361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:52.469361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:52.469361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:52.469361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:52.469361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:52.469361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:52.469361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:52.469361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:29:52.469361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:52.469361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:52.469361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:52.469361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:52.470199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:52.470199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:52.470199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:52.470199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:52.470199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:29:52.470199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:29:52.470199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:29:52.470199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:29:52.470199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:29:52.470199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:29:52.470199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:29:52.470199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:29:52.470199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 91/177] 10.222.0.13 () {56 vars in 1114 bytes} [Fri May 27 19:29:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:29:52.471647 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94253ac-ff5e-4c23-9f71-f46c8583a7cb None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:52.471847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba7a86b-6465-40be-afdb-06309a58af65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:52.472550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7a86b-6465-40be-afdb-06309a58af65 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:52.472832 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7a86b-6465-40be-afdb-06309a58af65 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:52.473184 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7a86b-6465-40be-afdb-06309a58af65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:52.552365 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7a86b-6465-40be-afdb-06309a58af65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:52.554082 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba7a86b-6465-40be-afdb-06309a58af65 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:52.557602 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:52.587728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94253ac-ff5e-4c23-9f71-f46c8583a7cb None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:52.628361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ca3b1-4fae-41b0-938d-99bc459833e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:52.629305 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ca3b1-4fae-41b0-938d-99bc459833e5 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:52.629593 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ca3b1-4fae-41b0-938d-99bc459833e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:52.629878 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ca3b1-4fae-41b0-938d-99bc459833e5 None admin] PATH_INFO: `/v3/users/02e527c314c64f03bbc6d51e8e772a09` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:52.631441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ca3b1-4fae-41b0-938d-99bc459833e5 None admin] RBAC: Authorizing `identity:get_user(user_id=02e527c314c64f03bbc6d51e8e772a09)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:52.633072 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ca3b1-4fae-41b0-938d-99bc459833e5 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:52.639644 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 92/178] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:29:52 2022] GET /identity/v3/users/02e527c314c64f03bbc6d51e8e772a09 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:52.639644 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e0cee-1a4b-42e0-ade1-ee76bd978410 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:52.639644 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e0cee-1a4b-42e0-ade1-ee76bd978410 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:52.639644 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e0cee-1a4b-42e0-ade1-ee76bd978410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:52.655868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-5a9e0cee-1a4b-42e0-ade1-ee76bd978410 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:52.657240 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-5a9e0cee-1a4b-42e0-ade1-ee76bd978410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:52.670665 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 87/179] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:29:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:29:52.670665 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66391e-9112-4945-8015-e5f7d52db739 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:52.670665 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66391e-9112-4945-8015-e5f7d52db739 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:52.670665 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66391e-9112-4945-8015-e5f7d52db739 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:52.670665 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 88/180] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:52.670665 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66391e-9112-4945-8015-e5f7d52db739 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:52.711992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:52.738378 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 93/181] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:52.739481 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e0cee-1a4b-42e0-ade1-ee76bd978410 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:52.785643 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66391e-9112-4945-8015-e5f7d52db739 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:52.788119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:52.824645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90c613-2f4a-4b5f-a54f-b3962ef769c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:52.825353 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90c613-2f4a-4b5f-a54f-b3962ef769c5 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:52.825637 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90c613-2f4a-4b5f-a54f-b3962ef769c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:52.825917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90c613-2f4a-4b5f-a54f-b3962ef769c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:52.827302 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad90c613-2f4a-4b5f-a54f-b3962ef769c5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:52.827574 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad90c613-2f4a-4b5f-a54f-b3962ef769c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:52.829406 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad90c613-2f4a-4b5f-a54f-b3962ef769c5 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:52.844186 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 89/182] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:29:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:52.844186 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec28ae-9a4c-4093-ac74-b868a88d12c7 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:52.844186 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec28ae-9a4c-4093-ac74-b868a88d12c7 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:52.844186 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec28ae-9a4c-4093-ac74-b868a88d12c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:52.861858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-2bec28ae-9a4c-4093-ac74-b868a88d12c7 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:52.863348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-2bec28ae-9a4c-4093-ac74-b868a88d12c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:52.864043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e0cee-1a4b-42e0-ade1-ee76bd978410 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:52.899404 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b8f1e-6bb6-43ce-bf35-8193acaf8ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:52.900061 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b8f1e-6bb6-43ce-bf35-8193acaf8ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:52.900348 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b8f1e-6bb6-43ce-bf35-8193acaf8ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:52.900624 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b8f1e-6bb6-43ce-bf35-8193acaf8ae7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:52.937895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 90/183] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:52.938485 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec28ae-9a4c-4093-ac74-b868a88d12c7 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:52.968878 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:52.994272 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b8f1e-6bb6-43ce-bf35-8193acaf8ae7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:52.996325 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b8f1e-6bb6-43ce-bf35-8193acaf8ae7 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:53.056520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:53.062920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec28ae-9a4c-4093-ac74-b868a88d12c7 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:53.100059 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2cecd0-1b74-4890-ac36-0069bb6ab10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:53.100987 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2cecd0-1b74-4890-ac36-0069bb6ab10d None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:53.101263 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2cecd0-1b74-4890-ac36-0069bb6ab10d None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:53.101520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2cecd0-1b74-4890-ac36-0069bb6ab10d None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:53.102719 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2cecd0-1b74-4890-ac36-0069bb6ab10d None admin] RBAC: Authorizing `identity:get_project(project_id=b30f38a32f6d4982bd829828ccac6fa7)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:53.104187 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2cecd0-1b74-4890-ac36-0069bb6ab10d None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:53.105972 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 91/184] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:29:52 2022] GET /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:53.106736 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2cecd0-1b74-4890-ac36-0069bb6ab10d None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:53.117737 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 94/185] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:29:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:29:53.118665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b8f1e-6bb6-43ce-bf35-8193acaf8ae7 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:53.205912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:53.224900 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2cecd0-1b74-4890-ac36-0069bb6ab10d None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:53.230396 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b8f1e-6bb6-43ce-bf35-8193acaf8ae7 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:53.261007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd16bc2-c33e-41eb-8a38-fbb9ff1649be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:53.261814 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd16bc2-c33e-41eb-8a38-fbb9ff1649be None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:53.262099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd16bc2-c33e-41eb-8a38-fbb9ff1649be None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:53.262378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd16bc2-c33e-41eb-8a38-fbb9ff1649be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:53.268610 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb927ef1-e183-444a-a197-9ce64a09666c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:53.269511 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb927ef1-e183-444a-a197-9ce64a09666c None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:53.269796 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb927ef1-e183-444a-a197-9ce64a09666c None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:53.270071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb927ef1-e183-444a-a197-9ce64a09666c None admin] PATH_INFO: `/v3/users/21fef53ebab34efcb73b2dd125d2c640` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:53.273544 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd16bc2-c33e-41eb-8a38-fbb9ff1649be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:29:53.281909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-edd16bc2-c33e-41eb-8a38-fbb9ff1649be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:29:53.281909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:53.281909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:53.281909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:53.281909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:53.281909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:53.281909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:53.281909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:53.281909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:53.281909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:53.281909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:53.281909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:53.281909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:29:53.281909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:29:53.281909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:29:53.281909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:29:53.282896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:29:53.282896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:29:53.282896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:29:53.282896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:29:53.282896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:53.282896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:53.282896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:53.282896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:53.282896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:53.282896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:53.282896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:53.282896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:29:53.282896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:53.282896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:53.282896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:53.282896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:53.283825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:53.283825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:53.283825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:53.283825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:53.283825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:29:53.283825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:29:53.283825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:29:53.283825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:29:53.283825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:29:53.283825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:29:53.283825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:29:53.283825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:29:53.283825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 92/186] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:29:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:29:53.284843 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd16bc2-c33e-41eb-8a38-fbb9ff1649be None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:53.284912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb927ef1-e183-444a-a197-9ce64a09666c None admin] RBAC: Authorizing `identity:get_user(user_id=21fef53ebab34efcb73b2dd125d2c640)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:53.286052 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb927ef1-e183-444a-a197-9ce64a09666c None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:53.287788 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 95/187] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:29:53 2022] GET /identity/v3/users/21fef53ebab34efcb73b2dd125d2c640 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:53.287788 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb927ef1-e183-444a-a197-9ce64a09666c None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:53.314432 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:53.389284 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd16bc2-c33e-41eb-8a38-fbb9ff1649be None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:53.392297 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb927ef1-e183-444a-a197-9ce64a09666c None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:53.424058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59087bc-ee8b-45b0-bcab-ae923aacae5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:53.424856 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59087bc-ee8b-45b0-bcab-ae923aacae5c None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:53.425170 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59087bc-ee8b-45b0-bcab-ae923aacae5c None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:53.425448 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59087bc-ee8b-45b0-bcab-ae923aacae5c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:53.426600 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59087bc-ee8b-45b0-bcab-ae923aacae5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:53.426778 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36b779-baf0-454a-b976-dc6232f2376b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:53.427439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36b779-baf0-454a-b976-dc6232f2376b None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:53.427727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36b779-baf0-454a-b976-dc6232f2376b None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:53.428010 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36b779-baf0-454a-b976-dc6232f2376b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:53.428096 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59087bc-ee8b-45b0-bcab-ae923aacae5c None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:53.428838 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36b779-baf0-454a-b976-dc6232f2376b None admin] RBAC: Adding query filter params (role.id=6505bbb129404436966ee59aa7a01d8e, scope.project.id=b30f38a32f6d4982bd829828ccac6fa7, user.id=736ecd08572d41d59ba0fe2db80e313a) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:53.429187 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36b779-baf0-454a-b976-dc6232f2376b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:53.429704 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 93/188] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:29:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:53.430473 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59087bc-ee8b-45b0-bcab-ae923aacae5c None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:53.430632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd36b779-baf0-454a-b976-dc6232f2376b None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:53.432516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:53.437666 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 96/189] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:29:53 2022] GET /identity/v3/role_assignments?user.id=736ecd08572d41d59ba0fe2db80e313a&scope.project.id=b30f38a32f6d4982bd829828ccac6fa7&role.id=6505bbb129404436966ee59aa7a01d8e => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:53.438956 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106fe29-f0fc-4262-892a-8e28a1f4ba2f None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:53.439341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106fe29-f0fc-4262-892a-8e28a1f4ba2f None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:53.439626 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106fe29-f0fc-4262-892a-8e28a1f4ba2f None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:53.440908 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 97/190] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:53.440908 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106fe29-f0fc-4262-892a-8e28a1f4ba2f None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:53.557385 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:53.570591 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59087bc-ee8b-45b0-bcab-ae923aacae5c None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:53.584013 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106fe29-f0fc-4262-892a-8e28a1f4ba2f None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:53.623468 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2386e-3d31-40ca-b19f-d0a95751159a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:53.624546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2386e-3d31-40ca-b19f-d0a95751159a None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:53.625272 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2386e-3d31-40ca-b19f-d0a95751159a None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:53.625272 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e2386e-3d31-40ca-b19f-d0a95751159a None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:53.627183 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2386e-3d31-40ca-b19f-d0a95751159a None admin] RBAC: Authorizing `identity:get_project(project_id=b30f38a32f6d4982bd829828ccac6fa7)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:53.627851 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e2386e-3d31-40ca-b19f-d0a95751159a None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:53.629383 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46fc68-3184-4e4d-b395-7c5cb15f5ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:53.631190 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 94/191] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:29:53 2022] GET /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:53.631291 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46fc68-3184-4e4d-b395-7c5cb15f5ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:53.631366 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2386e-3d31-40ca-b19f-d0a95751159a None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:53.631433 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46fc68-3184-4e4d-b395-7c5cb15f5ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:53.631433 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46fc68-3184-4e4d-b395-7c5cb15f5ce2 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:53.631564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46fc68-3184-4e4d-b395-7c5cb15f5ce2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:53.631779 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46fc68-3184-4e4d-b395-7c5cb15f5ce2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:53.633261 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46fc68-3184-4e4d-b395-7c5cb15f5ce2 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:53.648560 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 98/192] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:29:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:53.650451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46fc68-3184-4e4d-b395-7c5cb15f5ce2 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:53.734695 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:53.773056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e2386e-3d31-40ca-b19f-d0a95751159a None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:53.780751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46fc68-3184-4e4d-b395-7c5cb15f5ce2 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:53.812558 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c8727-2582-41c9-8fc6-9c132e5b446f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:53.813274 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c8727-2582-41c9-8fc6-9c132e5b446f None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:53.813562 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c8727-2582-41c9-8fc6-9c132e5b446f None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:53.813841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286c8727-2582-41c9-8fc6-9c132e5b446f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:53.819008 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e782d48-ba9c-4779-abe0-0afe69e844b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:53.819704 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e782d48-ba9c-4779-abe0-0afe69e844b7 None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:53.820023 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e782d48-ba9c-4779-abe0-0afe69e844b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:53.820311 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e782d48-ba9c-4779-abe0-0afe69e844b7 None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:53.821079 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e782d48-ba9c-4779-abe0-0afe69e844b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:53.822677 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e782d48-ba9c-4779-abe0-0afe69e844b7 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:53.846868 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:53.901869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c8727-2582-41c9-8fc6-9c132e5b446f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:53.903382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286c8727-2582-41c9-8fc6-9c132e5b446f None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:53.956083 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 99/193] 10.222.0.13 () {60 vars in 1132 bytes} [Fri May 27 19:29:53 2022] POST /identity/v3/users => generated 297 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:29:53.957462 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a556b6-60e8-4d6d-8d4a-b6d23410ee5a None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:53.957754 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a556b6-60e8-4d6d-8d4a-b6d23410ee5a None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:53.958032 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a556b6-60e8-4d6d-8d4a-b6d23410ee5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:53.979640 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-e2a556b6-60e8-4d6d-8d4a-b6d23410ee5a None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:53.981084 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-e2a556b6-60e8-4d6d-8d4a-b6d23410ee5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:54.033721 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 95/194] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:29:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:29:54.034477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c8727-2582-41c9-8fc6-9c132e5b446f None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:54.062839 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 100/195] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:54.070175 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a556b6-60e8-4d6d-8d4a-b6d23410ee5a None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:54.112104 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:54.125223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:54.164638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286c8727-2582-41c9-8fc6-9c132e5b446f None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:54.189685 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a556b6-60e8-4d6d-8d4a-b6d23410ee5a None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:54.212432 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6e6db-214d-4e3d-bc48-e0232271204e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:54.213143 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6e6db-214d-4e3d-bc48-e0232271204e None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:54.213431 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6e6db-214d-4e3d-bc48-e0232271204e None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:54.213709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6e6db-214d-4e3d-bc48-e0232271204e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:54.215070 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6e6db-214d-4e3d-bc48-e0232271204e None admin] RBAC: Adding query filter params (role.id=eda4ac05bc364224a7f319e7ded54073, scope.project.id=b30f38a32f6d4982bd829828ccac6fa7, user.id=02e527c314c64f03bbc6d51e8e772a09) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:54.215381 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6e6db-214d-4e3d-bc48-e0232271204e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:54.217774 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6e6db-214d-4e3d-bc48-e0232271204e None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:54.226433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 96/196] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:29:54 2022] GET /identity/v3/role_assignments?user.id=02e527c314c64f03bbc6d51e8e772a09&scope.project.id=b30f38a32f6d4982bd829828ccac6fa7&role.id=eda4ac05bc364224a7f319e7ded54073 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:54.227665 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6e6db-214d-4e3d-bc48-e0232271204e None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:54.230350 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfeead-1f38-44f6-a7ff-6f78575b12fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:54.231253 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfeead-1f38-44f6-a7ff-6f78575b12fe None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:54.231546 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfeead-1f38-44f6-a7ff-6f78575b12fe None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:54.231822 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfeead-1f38-44f6-a7ff-6f78575b12fe None admin] PATH_INFO: `/v3/users/21fef53ebab34efcb73b2dd125d2c640` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:54.233596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfeead-1f38-44f6-a7ff-6f78575b12fe None admin] RBAC: Authorizing `identity:get_user(user_id=21fef53ebab34efcb73b2dd125d2c640)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:54.235120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfeead-1f38-44f6-a7ff-6f78575b12fe None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:54.236983 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 101/197] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:29:54 2022] GET /identity/v3/users/21fef53ebab34efcb73b2dd125d2c640 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:54.237812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfeead-1f38-44f6-a7ff-6f78575b12fe None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:54.401325 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6e6db-214d-4e3d-bc48-e0232271204e None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:54.405008 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfeead-1f38-44f6-a7ff-6f78575b12fe None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:54.426138 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:54.442972 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:54.450066 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f927865-f663-4a4a-b282-9293eeac2d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:54.451798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f927865-f663-4a4a-b282-9293eeac2d2e None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:54.452286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f927865-f663-4a4a-b282-9293eeac2d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:54.452758 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f927865-f663-4a4a-b282-9293eeac2d2e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:54.458759 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd4d5f-a55b-4242-b7d6-318c005b3da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:54.459775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd4d5f-a55b-4242-b7d6-318c005b3da7 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:54.460071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd4d5f-a55b-4242-b7d6-318c005b3da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:54.460356 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd4d5f-a55b-4242-b7d6-318c005b3da7 None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:54.462234 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd4d5f-a55b-4242-b7d6-318c005b3da7 None admin] RBAC: Authorizing `identity:get_project(project_id=7abea439f4624f6680796de4a97fc13b)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:54.463719 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bd4d5f-a55b-4242-b7d6-318c005b3da7 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:54.464799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f927865-f663-4a4a-b282-9293eeac2d2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:29:54.466034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 97/198] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:29:54 2022] GET /identity/v3/projects/7abea439f4624f6680796de4a97fc13b => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:54.469781 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd4d5f-a55b-4242-b7d6-318c005b3da7 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:54.476623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-1f927865-f663-4a4a-b282-9293eeac2d2e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 19:29:54.476623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:54.476623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:54.476623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:54.476623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:54.476623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:54.476623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:54.476623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:54.476623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:54.476623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:54.476623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:54.476623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:54.476623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:29:54.476623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:29:54.476623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:29:54.476623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:29:54.479105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:29:54.479105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:29:54.479105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:29:54.479105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:29:54.479105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:54.479105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:54.479105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:54.479105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:54.479105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:54.479105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:54.479105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:54.479105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:29:54.479105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:54.479105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:54.479105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:54.479105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:54.480080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:54.480080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:54.480080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:54.480080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:54.480080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:29:54.480080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:29:54.480080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:29:54.480080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:29:54.480080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:29:54.480080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:29:54.480080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 19:29:54.480080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:29:54.482829 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 102/199] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:29:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:29:54.483669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f927865-f663-4a4a-b282-9293eeac2d2e None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:54.605209 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bd4d5f-a55b-4242-b7d6-318c005b3da7 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:54.619477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f927865-f663-4a4a-b282-9293eeac2d2e None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:54.657644 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b568cfa-3853-4602-85c2-fe027e5370a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:54.658561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b568cfa-3853-4602-85c2-fe027e5370a2 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:54.658841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b568cfa-3853-4602-85c2-fe027e5370a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:54.659112 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b568cfa-3853-4602-85c2-fe027e5370a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:54.669268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54195160-11dc-4491-8ffa-373c411b18b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:54.670551 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54195160-11dc-4491-8ffa-373c411b18b3 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:54.670831 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54195160-11dc-4491-8ffa-373c411b18b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:54.671108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54195160-11dc-4491-8ffa-373c411b18b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:54.672001 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54195160-11dc-4491-8ffa-373c411b18b3 None admin] RBAC: Adding query filter params (role.id=eda4ac05bc364224a7f319e7ded54073, scope.project.id=7abea439f4624f6680796de4a97fc13b, user.id=21fef53ebab34efcb73b2dd125d2c640) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:54.672316 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54195160-11dc-4491-8ffa-373c411b18b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:54.674406 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54195160-11dc-4491-8ffa-373c411b18b3 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:54.683205 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 103/200] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:29:54 2022] GET /identity/v3/role_assignments?user.id=21fef53ebab34efcb73b2dd125d2c640&scope.project.id=7abea439f4624f6680796de4a97fc13b&role.id=eda4ac05bc364224a7f319e7ded54073 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:29:54.684592 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54195160-11dc-4491-8ffa-373c411b18b3 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:54.736390 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:54.750443 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:54.756163 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b568cfa-3853-4602-85c2-fe027e5370a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:54.758356 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b568cfa-3853-4602-85c2-fe027e5370a2 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:54.833449 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54195160-11dc-4491-8ffa-373c411b18b3 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:54.910164 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6997a-94bc-4ee2-9ab3-6d5b87db1f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:54.910771 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6997a-94bc-4ee2-9ab3-6d5b87db1f69 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:54.911057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6997a-94bc-4ee2-9ab3-6d5b87db1f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:54.911334 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6997a-94bc-4ee2-9ab3-6d5b87db1f69 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:54.912099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6997a-94bc-4ee2-9ab3-6d5b87db1f69 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:54.912405 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6997a-94bc-4ee2-9ab3-6d5b87db1f69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:54.913857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6997a-94bc-4ee2-9ab3-6d5b87db1f69 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:54.918262 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 98/201] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:29:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:29:54.924789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b568cfa-3853-4602-85c2-fe027e5370a2 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:54.939198 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 104/202] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:29:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:54.949557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6997a-94bc-4ee2-9ab3-6d5b87db1f69 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:55.035717 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:55.045454 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b568cfa-3853-4602-85c2-fe027e5370a2 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:55.082234 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57545656-a659-4cdf-9aa2-235da694afcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:55.083103 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57545656-a659-4cdf-9aa2-235da694afcf None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:55.083382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57545656-a659-4cdf-9aa2-235da694afcf None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:55.083657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57545656-a659-4cdf-9aa2-235da694afcf None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:55.084984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57545656-a659-4cdf-9aa2-235da694afcf None admin] RBAC: Authorizing `identity:get_project(project_id=7abea439f4624f6680796de4a97fc13b)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:55.086454 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57545656-a659-4cdf-9aa2-235da694afcf None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:55.088281 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 99/203] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:29:54 2022] GET /identity/v3/projects/7abea439f4624f6680796de4a97fc13b => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:29:55.095333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57545656-a659-4cdf-9aa2-235da694afcf None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:55.102037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:55.109891 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6997a-94bc-4ee2-9ab3-6d5b87db1f69 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:55.192676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1ca68-dc85-4f49-9b7f-2f494f2e5e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:55.193526 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1ca68-dc85-4f49-9b7f-2f494f2e5e09 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:55.193808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1ca68-dc85-4f49-9b7f-2f494f2e5e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:55.194084 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1ca68-dc85-4f49-9b7f-2f494f2e5e09 None admin] PATH_INFO: `/v3/users/e00abb1711cb497187586e15b695c7d0` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:55.195456 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1ca68-dc85-4f49-9b7f-2f494f2e5e09 None admin] RBAC: Authorizing `identity:get_user(user_id=e00abb1711cb497187586e15b695c7d0)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:55.197153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1ca68-dc85-4f49-9b7f-2f494f2e5e09 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:55.199010 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 105/204] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:29:54 2022] GET /identity/v3/users/e00abb1711cb497187586e15b695c7d0 => generated 289 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:55.205715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1ca68-dc85-4f49-9b7f-2f494f2e5e09 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:55.258591 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57545656-a659-4cdf-9aa2-235da694afcf None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:55.328516 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40db4c-f9df-4166-90f1-72aa9f532618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:55.329409 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40db4c-f9df-4166-90f1-72aa9f532618 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:55.329692 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40db4c-f9df-4166-90f1-72aa9f532618 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:55.329967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff40db4c-f9df-4166-90f1-72aa9f532618 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:55.330807 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40db4c-f9df-4166-90f1-72aa9f532618 None admin] RBAC: Adding query filter params (role.id=6505bbb129404436966ee59aa7a01d8e, scope.project.id=7abea439f4624f6680796de4a97fc13b, user.id=21fef53ebab34efcb73b2dd125d2c640) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:55.331115 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40db4c-f9df-4166-90f1-72aa9f532618 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:55.332717 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff40db4c-f9df-4166-90f1-72aa9f532618 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:55.357603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 100/205] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:29:55 2022] GET /identity/v3/role_assignments?user.id=21fef53ebab34efcb73b2dd125d2c640&scope.project.id=7abea439f4624f6680796de4a97fc13b&role.id=6505bbb129404436966ee59aa7a01d8e => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:55.407652 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1ca68-dc85-4f49-9b7f-2f494f2e5e09 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:55.468219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798780b8-ec62-45f6-8d83-e68158364f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:55.468907 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798780b8-ec62-45f6-8d83-e68158364f16 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:55.469233 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798780b8-ec62-45f6-8d83-e68158364f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:55.469516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798780b8-ec62-45f6-8d83-e68158364f16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:55.518160 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:55.571325 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798780b8-ec62-45f6-8d83-e68158364f16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:55.572844 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798780b8-ec62-45f6-8d83-e68158364f16 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:55.692328 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 106/206] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:29:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 19:29:55.697843 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40db4c-f9df-4166-90f1-72aa9f532618 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:55.745955 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:55.836282 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff40db4c-f9df-4166-90f1-72aa9f532618 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:55.876233 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736412dd-7965-4c0e-bb59-c9da89d1a3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:55.877146 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736412dd-7965-4c0e-bb59-c9da89d1a3a1 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:55.877436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736412dd-7965-4c0e-bb59-c9da89d1a3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:55.877719 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736412dd-7965-4c0e-bb59-c9da89d1a3a1 None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:55.881694 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736412dd-7965-4c0e-bb59-c9da89d1a3a1 None admin] RBAC: Authorizing `identity:get_project(project_id=b30f38a32f6d4982bd829828ccac6fa7)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:55.883175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736412dd-7965-4c0e-bb59-c9da89d1a3a1 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:55.885304 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 101/207] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:29:55 2022] GET /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:55.891590 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798780b8-ec62-45f6-8d83-e68158364f16 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:55.990247 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:56.025874 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798780b8-ec62-45f6-8d83-e68158364f16 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:56.076600 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcffa80e-c12c-41c3-b2d7-d444a5f635b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:56.077296 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffa80e-c12c-41c3-b2d7-d444a5f635b1 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:56.077845 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffa80e-c12c-41c3-b2d7-d444a5f635b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:56.078325 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcffa80e-c12c-41c3-b2d7-d444a5f635b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:56.079364 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcffa80e-c12c-41c3-b2d7-d444a5f635b1 None admin] RBAC: Adding query filter params (role.id=9170e7d540b446f3b5507688ca335ecc, scope.project.id=b30f38a32f6d4982bd829828ccac6fa7, user.id=e00abb1711cb497187586e15b695c7d0) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:56.079899 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcffa80e-c12c-41c3-b2d7-d444a5f635b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:56.081601 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcffa80e-c12c-41c3-b2d7-d444a5f635b1 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:56.088300 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 107/208] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:29:55 2022] GET /identity/v3/role_assignments?user.id=e00abb1711cb497187586e15b695c7d0&scope.project.id=b30f38a32f6d4982bd829828ccac6fa7&role.id=9170e7d540b446f3b5507688ca335ecc => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:56.853810 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00baf2e-f1e3-4749-8bcd-68c8e4c52a0c None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:56.854302 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00baf2e-f1e3-4749-8bcd-68c8e4c52a0c None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:56.854302 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00baf2e-f1e3-4749-8bcd-68c8e4c52a0c None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:56.855986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 102/209] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:56.861848 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c5ed3-cc92-4532-996a-4f931a04c70b None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:56.862422 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c5ed3-cc92-4532-996a-4f931a04c70b None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:56.862940 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c5ed3-cc92-4532-996a-4f931a04c70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:56.869619 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d7654-f29f-422a-9376-c69e16571d85 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:56.870210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d7654-f29f-422a-9376-c69e16571d85 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:56.871583 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d7654-f29f-422a-9376-c69e16571d85 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:56.873622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 103/210] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 19:29:56.882543 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-a11c5ed3-cc92-4532-996a-4f931a04c70b None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:56.883348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df6ad5-55ea-4999-852c-161ae396ab18 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:56.883712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df6ad5-55ea-4999-852c-161ae396ab18 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:56.883815 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-a11c5ed3-cc92-4532-996a-4f931a04c70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:56.884073 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df6ad5-55ea-4999-852c-161ae396ab18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:56.901683 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-23df6ad5-55ea-4999-852c-161ae396ab18 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:56.903220 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-23df6ad5-55ea-4999-852c-161ae396ab18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:56.976403 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 108/211] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:56.998643 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 104/212] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 19:29:57.001583 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c5ed3-cc92-4532-996a-4f931a04c70b None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:57.026885 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df6ad5-55ea-4999-852c-161ae396ab18 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:57.058982 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:57.068243 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:57.111013 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c5ed3-cc92-4532-996a-4f931a04c70b None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:57.150118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2539e1-2aab-43c3-b39b-74b9ebc66675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:57.151291 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2539e1-2aab-43c3-b39b-74b9ebc66675 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:57.151875 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2539e1-2aab-43c3-b39b-74b9ebc66675 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:57.152383 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2539e1-2aab-43c3-b39b-74b9ebc66675 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:57.154391 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2539e1-2aab-43c3-b39b-74b9ebc66675 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:57.156043 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df6ad5-55ea-4999-852c-161ae396ab18 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:57.156536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2539e1-2aab-43c3-b39b-74b9ebc66675 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:57.158726 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 109/213] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:29:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:57.164467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2539e1-2aab-43c3-b39b-74b9ebc66675 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:57.196317 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ee6c1-9e2c-4ea7-ab2d-7a031413b301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:57.197470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ee6c1-9e2c-4ea7-ab2d-7a031413b301 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:57.197899 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ee6c1-9e2c-4ea7-ab2d-7a031413b301 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:57.198319 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ee6c1-9e2c-4ea7-ab2d-7a031413b301 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:57.211060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ee6c1-9e2c-4ea7-ab2d-7a031413b301 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:29:57.220362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-9b8ee6c1-9e2c-4ea7-ab2d-7a031413b301 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 19:29:57.220362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:57.220362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:57.220362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:57.220362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:57.220362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:57.220362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:57.220362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:57.220362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:57.220362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:57.220362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:57.220362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:57.220362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:29:57.220362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:29:57.220362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:29:57.220362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:29:57.221369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:29:57.221369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:29:57.221369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:29:57.221369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:29:57.221369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:57.221369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:57.221369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:57.221369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:57.221369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:57.221369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:57.221369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:57.221369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:29:57.221369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:57.221369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:57.221369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:57.221369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:57.222167 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:57.222167 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:57.222167 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:57.222167 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:57.222167 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:29:57.222167 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:29:57.222167 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:29:57.222167 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:29:57.222167 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:29:57.222167 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:29:57.222167 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 19:29:57.222167 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:29:57.224755 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 105/214] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:29:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:29:57.230859 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ee6c1-9e2c-4ea7-ab2d-7a031413b301 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:57.276935 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2539e1-2aab-43c3-b39b-74b9ebc66675 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:57.308047 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:57.315030 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557dcad1-de59-4468-b07b-30d38f268a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:57.315808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dcad1-de59-4468-b07b-30d38f268a18 None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:57.316098 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dcad1-de59-4468-b07b-30d38f268a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:57.316374 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557dcad1-de59-4468-b07b-30d38f268a18 None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:57.317120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557dcad1-de59-4468-b07b-30d38f268a18 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:57.325619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557dcad1-de59-4468-b07b-30d38f268a18 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:57.325753 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:57.346048 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ee6c1-9e2c-4ea7-ab2d-7a031413b301 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:57.382602 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c66f31-4b33-4339-b8cf-b1e11e8cd0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:57.383209 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c66f31-4b33-4339-b8cf-b1e11e8cd0c0 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:57.383495 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c66f31-4b33-4339-b8cf-b1e11e8cd0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:57.383774 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c66f31-4b33-4339-b8cf-b1e11e8cd0c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:57.384533 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c66f31-4b33-4339-b8cf-b1e11e8cd0c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:57.384845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c66f31-4b33-4339-b8cf-b1e11e8cd0c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:57.386295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c66f31-4b33-4339-b8cf-b1e11e8cd0c0 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:57.396982 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 106/215] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:29:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:29:57.403268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c66f31-4b33-4339-b8cf-b1e11e8cd0c0 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:57.450080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 110/216] 10.222.0.13 () {60 vars in 1132 bytes} [Fri May 27 19:29:57 2022] POST /identity/v3/users => generated 297 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:29:57.450080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885405ef-a695-4b21-8baf-ab07e69bccab None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:57.450080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885405ef-a695-4b21-8baf-ab07e69bccab None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:57.450080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885405ef-a695-4b21-8baf-ab07e69bccab None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:57.450080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 111/217] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:57.453458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cd404-6939-40cc-b731-3ce4dff69906 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:57.453768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cd404-6939-40cc-b731-3ce4dff69906 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:57.454135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638cd404-6939-40cc-b731-3ce4dff69906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:57.478881 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-638cd404-6939-40cc-b731-3ce4dff69906 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:57.482379 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-638cd404-6939-40cc-b731-3ce4dff69906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:57.543700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c66f31-4b33-4339-b8cf-b1e11e8cd0c0 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:57.584230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6a7f4-1b90-4a8f-a11b-14ebd2285d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:57.585209 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6a7f4-1b90-4a8f-a11b-14ebd2285d14 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:57.585491 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6a7f4-1b90-4a8f-a11b-14ebd2285d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:57.585788 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6a7f4-1b90-4a8f-a11b-14ebd2285d14 None admin] PATH_INFO: `/v3/users/b591c0899b2f426e92cb2b7c82cb80a7` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:57.601036 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 112/218] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:57.612652 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a6a7f4-1b90-4a8f-a11b-14ebd2285d14 None admin] RBAC: Authorizing `identity:get_user(user_id=b591c0899b2f426e92cb2b7c82cb80a7)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:57.614495 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a6a7f4-1b90-4a8f-a11b-14ebd2285d14 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:57.616800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 107/219] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:29:57 2022] GET /identity/v3/users/b591c0899b2f426e92cb2b7c82cb80a7 => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:29:57.622070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638cd404-6939-40cc-b731-3ce4dff69906 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:57.624117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6a7f4-1b90-4a8f-a11b-14ebd2285d14 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:57.627648 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:57.781199 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6a7f4-1b90-4a8f-a11b-14ebd2285d14 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:57.784746 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638cd404-6939-40cc-b731-3ce4dff69906 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:57.811411 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:57.828050 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7486842-34ba-4f3a-bc22-089cb8621781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:57.828537 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7486842-34ba-4f3a-bc22-089cb8621781 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:57.828825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7486842-34ba-4f3a-bc22-089cb8621781 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:57.829138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7486842-34ba-4f3a-bc22-089cb8621781 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:57.842141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab12df-4028-476e-a32b-8ef8c280c2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:57.842895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab12df-4028-476e-a32b-8ef8c280c2ea None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:57.843583 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab12df-4028-476e-a32b-8ef8c280c2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:57.843708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab12df-4028-476e-a32b-8ef8c280c2ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:57.849625 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7486842-34ba-4f3a-bc22-089cb8621781 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:29:57.858035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-e7486842-34ba-4f3a-bc22-089cb8621781 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:29:57.858035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:57.858035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:57.858035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:57.858035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:57.858035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:57.858035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:57.858035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:57.858035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:57.858035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:57.858035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:57.858035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:57.858035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:29:57.858035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:29:57.858035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:29:57.858035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:29:57.859632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:29:57.859632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:29:57.859632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:29:57.859632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:29:57.859632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:57.859632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:57.859632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:57.859632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:57.859632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:57.859632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:57.859632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:57.859632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:29:57.859632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:57.859632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:57.859632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:57.859632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:57.861457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:57.861457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:57.861457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:57.861457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:57.861457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:29:57.861457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:29:57.861457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:29:57.861457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:29:57.861457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:29:57.861457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:29:57.861457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:29:57.861457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:29:57.861457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 108/220] 10.222.0.13 () {56 vars in 1114 bytes} [Fri May 27 19:29:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:29:57.865103 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7486842-34ba-4f3a-bc22-089cb8621781 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:57.934709 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab12df-4028-476e-a32b-8ef8c280c2ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:57.936972 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:57.938135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faab12df-4028-476e-a32b-8ef8c280c2ea None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:57.978686 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7486842-34ba-4f3a-bc22-089cb8621781 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:58.015244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e149c48-932d-4820-b9ea-8286e661985c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:58.016204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e149c48-932d-4820-b9ea-8286e661985c None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:58.017439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e149c48-932d-4820-b9ea-8286e661985c None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:58.017439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e149c48-932d-4820-b9ea-8286e661985c None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:58.018461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e149c48-932d-4820-b9ea-8286e661985c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:58.019024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e149c48-932d-4820-b9ea-8286e661985c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:58.021417 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e149c48-932d-4820-b9ea-8286e661985c None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:58.031605 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 109/221] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:29:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:29:58.037044 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e149c48-932d-4820-b9ea-8286e661985c None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:58.053636 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 113/222] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:29:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:29:58.060024 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8b75e-450d-4032-abd8-c7c28ba68746 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:58.060632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8b75e-450d-4032-abd8-c7c28ba68746 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:58.061152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8b75e-450d-4032-abd8-c7c28ba68746 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:58.063119 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 114/223] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:29:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:29:58.064114 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8b75e-450d-4032-abd8-c7c28ba68746 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:58.086007 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:58.163397 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e149c48-932d-4820-b9ea-8286e661985c None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:58.191153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8b75e-450d-4032-abd8-c7c28ba68746 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:58.200901 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:58.216016 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f12bf-eb60-48a8-bd8a-a65c0c82e8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:58.217759 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f12bf-eb60-48a8-bd8a-a65c0c82e8fb None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:58.218679 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f12bf-eb60-48a8-bd8a-a65c0c82e8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:58.219425 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f12bf-eb60-48a8-bd8a-a65c0c82e8fb None admin] PATH_INFO: `/v3/users/21fef53ebab34efcb73b2dd125d2c640` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:58.221463 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f12bf-eb60-48a8-bd8a-a65c0c82e8fb None admin] RBAC: Authorizing `identity:get_user(user_id=21fef53ebab34efcb73b2dd125d2c640)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:58.223650 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f12bf-eb60-48a8-bd8a-a65c0c82e8fb None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:58.226805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 110/224] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:29:58 2022] GET /identity/v3/users/21fef53ebab34efcb73b2dd125d2c640 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:58.228809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339785ea-4907-4efd-a59f-ef83f1797dc7 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:58.229314 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339785ea-4907-4efd-a59f-ef83f1797dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:58.229675 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3388b-28ef-41f9-aa0f-dc0c3c11af56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:58.230016 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339785ea-4907-4efd-a59f-ef83f1797dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:58.231075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3388b-28ef-41f9-aa0f-dc0c3c11af56 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:58.231435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3388b-28ef-41f9-aa0f-dc0c3c11af56 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:58.232373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a3388b-28ef-41f9-aa0f-dc0c3c11af56 None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:58.233948 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3388b-28ef-41f9-aa0f-dc0c3c11af56 None admin] RBAC: Authorizing `identity:get_project(project_id=7abea439f4624f6680796de4a97fc13b)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:58.235914 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a3388b-28ef-41f9-aa0f-dc0c3c11af56 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:58.242718 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 115/225] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:29:58 2022] GET /identity/v3/projects/7abea439f4624f6680796de4a97fc13b => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:58.242718 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3388b-28ef-41f9-aa0f-dc0c3c11af56 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:58.258101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-339785ea-4907-4efd-a59f-ef83f1797dc7 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:29:58.260251 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-339785ea-4907-4efd-a59f-ef83f1797dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:29:58.355374 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:58.363695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 111/226] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:29:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:29:58.364833 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339785ea-4907-4efd-a59f-ef83f1797dc7 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:58.391892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a3388b-28ef-41f9-aa0f-dc0c3c11af56 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:58.439377 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7e802-614e-42f9-981c-8b98cc1928ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:58.440263 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7e802-614e-42f9-981c-8b98cc1928ba None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:58.440686 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7e802-614e-42f9-981c-8b98cc1928ba None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:58.445224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a7e802-614e-42f9-981c-8b98cc1928ba None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:58.445224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a7e802-614e-42f9-981c-8b98cc1928ba None admin] RBAC: Authorizing `identity:get_project(project_id=7abea439f4624f6680796de4a97fc13b)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:58.445998 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a7e802-614e-42f9-981c-8b98cc1928ba None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:58.449048 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 116/227] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:29:58 2022] GET /identity/v3/projects/7abea439f4624f6680796de4a97fc13b => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:58.451423 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7e802-614e-42f9-981c-8b98cc1928ba None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:58.549888 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339785ea-4907-4efd-a59f-ef83f1797dc7 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:58.575026 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:58.587315 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a7e802-614e-42f9-981c-8b98cc1928ba None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:58.589129 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db06af2-bb25-4deb-9be9-45877cc22845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:58.589984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db06af2-bb25-4deb-9be9-45877cc22845 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:58.590417 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db06af2-bb25-4deb-9be9-45877cc22845 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:58.590838 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db06af2-bb25-4deb-9be9-45877cc22845 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:58.592851 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db06af2-bb25-4deb-9be9-45877cc22845 None admin] RBAC: Adding query filter params (role.id=cb43a575e47147c799eec2f362f4fe0e, scope.project.id=7abea439f4624f6680796de4a97fc13b, user.id=b591c0899b2f426e92cb2b7c82cb80a7) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:58.593347 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db06af2-bb25-4deb-9be9-45877cc22845 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:58.595071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db06af2-bb25-4deb-9be9-45877cc22845 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:58.601680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 112/228] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:29:58 2022] GET /identity/v3/role_assignments?user.id=b591c0899b2f426e92cb2b7c82cb80a7&scope.project.id=7abea439f4624f6680796de4a97fc13b&role.id=cb43a575e47147c799eec2f362f4fe0e => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:58.602691 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db06af2-bb25-4deb-9be9-45877cc22845 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:58.638682 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883bdae3-35b1-482b-815a-184645da6748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:58.639125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bdae3-35b1-482b-815a-184645da6748 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:58.639413 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bdae3-35b1-482b-815a-184645da6748 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:58.639741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bdae3-35b1-482b-815a-184645da6748 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:58.657163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883bdae3-35b1-482b-815a-184645da6748 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:29:58.667486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-883bdae3-35b1-482b-815a-184645da6748 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:29:58.667486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:29:58.667486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:29:58.667486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:29:58.667486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:29:58.667486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:29:58.667486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:29:58.667486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:29:58.667486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:29:58.667486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:29:58.667486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:29:58.667486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:29:58.667486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:29:58.667486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:29:58.667486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:29:58.667486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:29:58.668450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:29:58.668450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:29:58.668450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:29:58.668450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:29:58.668450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:29:58.668450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:29:58.668450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:29:58.668450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:29:58.668450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:29:58.668450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:29:58.668450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:29:58.668450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:29:58.668450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:29:58.668450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:29:58.668450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:29:58.668450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:29:58.670029 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:29:58.670029 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:29:58.670029 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:29:58.670029 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:29:58.670029 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:29:58.670029 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:29:58.670029 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:29:58.670029 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:29:58.670029 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:29:58.670029 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:29:58.670029 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:29:58.670029 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:29:58.676459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 117/229] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:29:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:29:58.682663 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883bdae3-35b1-482b-815a-184645da6748 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:58.729729 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:58.809353 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db06af2-bb25-4deb-9be9-45877cc22845 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:58.844420 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883bdae3-35b1-482b-815a-184645da6748 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:58.886952 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3d0ba-b8fc-497a-a3e9-0f18ddfeeaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:58.888520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3d0ba-b8fc-497a-a3e9-0f18ddfeeaca None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:58.889103 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3d0ba-b8fc-497a-a3e9-0f18ddfeeaca None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:58.889679 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3d0ba-b8fc-497a-a3e9-0f18ddfeeaca None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:58.890637 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3d0ba-b8fc-497a-a3e9-0f18ddfeeaca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:29:58.891212 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3d0ba-b8fc-497a-a3e9-0f18ddfeeaca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:58.892894 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e3d0ba-b8fc-497a-a3e9-0f18ddfeeaca None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:58.899779 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbacb35f-70ab-4b23-8c72-4f9fbe8e844b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:58.901339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbacb35f-70ab-4b23-8c72-4f9fbe8e844b None admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:58.901764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbacb35f-70ab-4b23-8c72-4f9fbe8e844b None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:58.902134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbacb35f-70ab-4b23-8c72-4f9fbe8e844b None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b/users/21fef53ebab34efcb73b2dd125d2c640/roles/eda4ac05bc364224a7f319e7ded54073` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:58.904595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 118/230] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:29:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:58.909820 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbacb35f-70ab-4b23-8c72-4f9fbe8e844b None admin] RBAC: Authorizing `identity:create_grant(project_id=7abea439f4624f6680796de4a97fc13b, user_id=21fef53ebab34efcb73b2dd125d2c640, role_id=eda4ac05bc364224a7f319e7ded54073)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:58.910745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3d0ba-b8fc-497a-a3e9-0f18ddfeeaca None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:58.911532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbacb35f-70ab-4b23-8c72-4f9fbe8e844b None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:58.991249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 113/231] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:29:58 2022] PUT /identity/v3/projects/7abea439f4624f6680796de4a97fc13b/users/21fef53ebab34efcb73b2dd125d2c640/roles/eda4ac05bc364224a7f319e7ded54073 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:29:59.016929 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:59.048251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3d0ba-b8fc-497a-a3e9-0f18ddfeeaca None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:59.101461 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad70afe-fb1f-4181-a049-ddc48eedb7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:59.103402 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad70afe-fb1f-4181-a049-ddc48eedb7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:59.103693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad70afe-fb1f-4181-a049-ddc48eedb7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:59.103974 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad70afe-fb1f-4181-a049-ddc48eedb7d0 None admin] PATH_INFO: `/v3/users/21fef53ebab34efcb73b2dd125d2c640` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:59.105396 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad70afe-fb1f-4181-a049-ddc48eedb7d0 None admin] RBAC: Authorizing `identity:get_user(user_id=21fef53ebab34efcb73b2dd125d2c640)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:59.107035 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad70afe-fb1f-4181-a049-ddc48eedb7d0 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:59.108928 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 119/232] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:29:58 2022] GET /identity/v3/users/21fef53ebab34efcb73b2dd125d2c640 => generated 283 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:59.114865 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbacb35f-70ab-4b23-8c72-4f9fbe8e844b None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:59.187952 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:29:59.269080 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbacb35f-70ab-4b23-8c72-4f9fbe8e844b None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:59.314092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692cab53-6d11-49e5-981a-9b1aa82e0958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:59.315336 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cab53-6d11-49e5-981a-9b1aa82e0958 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:59.315799 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cab53-6d11-49e5-981a-9b1aa82e0958 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:59.316251 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cab53-6d11-49e5-981a-9b1aa82e0958 None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:59.317629 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692cab53-6d11-49e5-981a-9b1aa82e0958 None admin] RBAC: Authorizing `identity:get_project(project_id=7abea439f4624f6680796de4a97fc13b)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:59.319291 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692cab53-6d11-49e5-981a-9b1aa82e0958 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:59.321310 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 114/233] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:29:59 2022] GET /identity/v3/projects/7abea439f4624f6680796de4a97fc13b => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:29:59.328574 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad70afe-fb1f-4181-a049-ddc48eedb7d0 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:29:59.463335 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad70afe-fb1f-4181-a049-ddc48eedb7d0 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:29:59.513622 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42728bb7-d8a6-4e86-98ab-b75c75bed95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:29:59.515188 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42728bb7-d8a6-4e86-98ab-b75c75bed95b None admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:29:59.515691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42728bb7-d8a6-4e86-98ab-b75c75bed95b None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:29:59.516031 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42728bb7-d8a6-4e86-98ab-b75c75bed95b None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b/users/21fef53ebab34efcb73b2dd125d2c640/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:29:59.518432 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42728bb7-d8a6-4e86-98ab-b75c75bed95b None admin] RBAC: Authorizing `identity:create_grant(project_id=7abea439f4624f6680796de4a97fc13b, user_id=21fef53ebab34efcb73b2dd125d2c640, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:29:59.524047 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42728bb7-d8a6-4e86-98ab-b75c75bed95b None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:29:59.646291 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 120/234] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:29:59 2022] PUT /identity/v3/projects/7abea439f4624f6680796de4a97fc13b/users/21fef53ebab34efcb73b2dd125d2c640/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:30:00.432374 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aded19-3cfc-4480-bfa0-1fd1127e8d51 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:00.432902 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aded19-3cfc-4480-bfa0-1fd1127e8d51 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:00.433035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aded19-3cfc-4480-bfa0-1fd1127e8d51 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:00.434338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 115/235] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:00.441530 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a7ca0-74ce-4e7c-ac01-6b1622c81b69 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:00.441992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a7ca0-74ce-4e7c-ac01-6b1622c81b69 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:00.442066 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a7ca0-74ce-4e7c-ac01-6b1622c81b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:00.454374 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:00.469966 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-1b0a7ca0-74ce-4e7c-ac01-6b1622c81b69 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:00.471176 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-1b0a7ca0-74ce-4e7c-ac01-6b1622c81b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:00.563660 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 121/236] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 19:30:00.570721 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9f33b-9e41-46ff-8e59-f3d56efd0c75 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:00.571015 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9f33b-9e41-46ff-8e59-f3d56efd0c75 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:00.571304 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9f33b-9e41-46ff-8e59-f3d56efd0c75 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:00.572683 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 116/237] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:00.578243 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afde39-f9e3-4584-b4d6-c8003f8a4128 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:00.578539 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afde39-f9e3-4584-b4d6-c8003f8a4128 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:00.578822 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afde39-f9e3-4584-b4d6-c8003f8a4128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:00.588724 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9f33b-9e41-46ff-8e59-f3d56efd0c75 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:00.602577 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-51afde39-f9e3-4584-b4d6-c8003f8a4128 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:00.603803 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-51afde39-f9e3-4584-b4d6-c8003f8a4128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:00.717020 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 122/238] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 19:30:00.775035 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51afde39-f9e3-4584-b4d6-c8003f8a4128 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:00.807847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9f33b-9e41-46ff-8e59-f3d56efd0c75 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:00.836216 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:00.872009 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:00.893674 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42b2d2-e824-47d0-a854-f63b6ff23735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:00.894835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42b2d2-e824-47d0-a854-f63b6ff23735 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:00.895326 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42b2d2-e824-47d0-a854-f63b6ff23735 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:00.895832 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df42b2d2-e824-47d0-a854-f63b6ff23735 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:00.897289 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42b2d2-e824-47d0-a854-f63b6ff23735 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:00.898957 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df42b2d2-e824-47d0-a854-f63b6ff23735 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:00.901112 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 117/239] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:00.906955 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42b2d2-e824-47d0-a854-f63b6ff23735 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:00.907391 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51afde39-f9e3-4584-b4d6-c8003f8a4128 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:00.944840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c628714-5071-4ec0-8dce-775038bd771e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:00.946121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c628714-5071-4ec0-8dce-775038bd771e None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:00.946409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c628714-5071-4ec0-8dce-775038bd771e None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:00.946688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c628714-5071-4ec0-8dce-775038bd771e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:00.959907 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c628714-5071-4ec0-8dce-775038bd771e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:00.969868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-6c628714-5071-4ec0-8dce-775038bd771e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 19:30:00.969868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:00.969868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:00.969868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:00.969868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:00.969868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:00.969868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:00.969868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:00.969868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:00.969868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:00.969868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:00.969868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:00.969868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:00.969868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:00.969868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:00.969868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:00.970812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:00.970812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:00.970812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:00.970812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:00.970812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:00.970812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:00.970812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:00.970812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:00.970812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:00.970812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:00.970812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:00.970812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:00.970812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:00.970812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:00.970812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:00.970812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:00.971692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:00.971692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:00.971692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:00.971692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:00.971692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:00.971692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:00.971692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:00.971692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:00.971692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:00.971692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:00.971692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 19:30:00.971692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:00.971692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 123/240] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:30:00.977006 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c628714-5071-4ec0-8dce-775038bd771e None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:01.092014 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c628714-5071-4ec0-8dce-775038bd771e None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:01.125063 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:01.156774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24d12c-6b6f-4cd1-953f-2dfa7544ab34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:01.157865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24d12c-6b6f-4cd1-953f-2dfa7544ab34 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:01.158354 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24d12c-6b6f-4cd1-953f-2dfa7544ab34 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:01.158934 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24d12c-6b6f-4cd1-953f-2dfa7544ab34 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:01.161750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24d12c-6b6f-4cd1-953f-2dfa7544ab34 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:01.162305 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24d12c-6b6f-4cd1-953f-2dfa7544ab34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:01.163938 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24d12c-6b6f-4cd1-953f-2dfa7544ab34 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:01.183958 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 124/241] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:30:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:01.190047 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24d12c-6b6f-4cd1-953f-2dfa7544ab34 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:01.195701 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df42b2d2-e824-47d0-a854-f63b6ff23735 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:01.258209 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768993f-a423-4b87-9bf9-86c4f77677af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:01.258811 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768993f-a423-4b87-9bf9-86c4f77677af None admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:01.259130 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768993f-a423-4b87-9bf9-86c4f77677af None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:01.259358 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768993f-a423-4b87-9bf9-86c4f77677af None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:01.260097 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768993f-a423-4b87-9bf9-86c4f77677af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:01.261587 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768993f-a423-4b87-9bf9-86c4f77677af None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:01.340308 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24d12c-6b6f-4cd1-953f-2dfa7544ab34 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:01.377779 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5866e5-f6d1-4c28-86a5-1db981715686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:01.378657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5866e5-f6d1-4c28-86a5-1db981715686 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:01.378984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5866e5-f6d1-4c28-86a5-1db981715686 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:01.379307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5866e5-f6d1-4c28-86a5-1db981715686 None admin] PATH_INFO: `/v3/users/ea4d03e2ac6c4426951e5cfee25db55d` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:01.381231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 118/242] 10.222.0.13 () {60 vars in 1132 bytes} [Fri May 27 19:30:00 2022] POST /identity/v3/users => generated 293 bytes in 475 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:30:01.383324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b2127-9ce5-4b9c-8916-aa95e53bd031 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:01.383844 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b2127-9ce5-4b9c-8916-aa95e53bd031 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:01.384339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b2127-9ce5-4b9c-8916-aa95e53bd031 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:01.385914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 119/243] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:01.394201 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479520a-7dee-4cdb-b38a-09ad58b4d0a5 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:01.394870 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479520a-7dee-4cdb-b38a-09ad58b4d0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:01.395352 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d479520a-7dee-4cdb-b38a-09ad58b4d0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:01.416714 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5866e5-f6d1-4c28-86a5-1db981715686 None admin] RBAC: Authorizing `identity:get_user(user_id=ea4d03e2ac6c4426951e5cfee25db55d)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:01.418682 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5866e5-f6d1-4c28-86a5-1db981715686 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:01.421260 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 125/244] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:30:01 2022] GET /identity/v3/users/ea4d03e2ac6c4426951e5cfee25db55d => generated 297 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:01.427838 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5866e5-f6d1-4c28-86a5-1db981715686 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:01.439818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-d479520a-7dee-4cdb-b38a-09ad58b4d0a5 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:01.441990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d479520a-7dee-4cdb-b38a-09ad58b4d0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:01.527641 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:01.533772 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:01.596969 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5866e5-f6d1-4c28-86a5-1db981715686 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:01.616731 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 120/245] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:01.636576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c08eb-bc10-4ee2-8174-5dbf43cec899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:01.637296 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c08eb-bc10-4ee2-8174-5dbf43cec899 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:01.637622 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c08eb-bc10-4ee2-8174-5dbf43cec899 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:01.637908 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41c08eb-bc10-4ee2-8174-5dbf43cec899 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:01.641088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479520a-7dee-4cdb-b38a-09ad58b4d0a5 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:01.717647 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c08eb-bc10-4ee2-8174-5dbf43cec899 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:01.719148 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41c08eb-bc10-4ee2-8174-5dbf43cec899 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:01.751579 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d479520a-7dee-4cdb-b38a-09ad58b4d0a5 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:01.771658 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:01.799948 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a77db17-860d-4efe-90ca-6b2e8a4711f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:01.800773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77db17-860d-4efe-90ca-6b2e8a4711f1 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:01.801146 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77db17-860d-4efe-90ca-6b2e8a4711f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:01.801534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a77db17-860d-4efe-90ca-6b2e8a4711f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:01.816649 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a77db17-860d-4efe-90ca-6b2e8a4711f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:01.824739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-9a77db17-860d-4efe-90ca-6b2e8a4711f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 19:30:01.824739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:01.824739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:01.824739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:01.824739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:01.824739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:01.824739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:01.824739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:01.824739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:01.824739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:01.824739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:01.824739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:01.824739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:01.824739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:01.824739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:01.824739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:01.826383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:01.826383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:01.826383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:01.826383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:01.826383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:01.826383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:01.826383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:01.826383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:01.826383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:01.826383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:01.826383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:01.826383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:01.826383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:01.826383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:01.826383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:01.826383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:01.827506 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:01.827506 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:01.827506 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:01.827506 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:01.827506 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:01.827506 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:01.827506 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:01.827506 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:01.827506 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:01.827506 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:01.827506 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 19:30:01.827506 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:01.828132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 121/246] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:30:01.835955 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a77db17-860d-4efe-90ca-6b2e8a4711f1 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:01.851461 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 126/247] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:30:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:30:01.858031 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c08eb-bc10-4ee2-8174-5dbf43cec899 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:01.882896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:01.983530 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41c08eb-bc10-4ee2-8174-5dbf43cec899 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:01.984537 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a77db17-860d-4efe-90ca-6b2e8a4711f1 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:02.032986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81993fb-86d1-46be-8fa9-8601fa7432a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:02.033368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f73a36-257d-46fb-b6e4-6da1b0967cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:02.033621 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81993fb-86d1-46be-8fa9-8601fa7432a4 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:02.033906 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81993fb-86d1-46be-8fa9-8601fa7432a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:02.034183 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81993fb-86d1-46be-8fa9-8601fa7432a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:02.034832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f73a36-257d-46fb-b6e4-6da1b0967cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:02.034932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81993fb-86d1-46be-8fa9-8601fa7432a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:02.035119 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f73a36-257d-46fb-b6e4-6da1b0967cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:02.035222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81993fb-86d1-46be-8fa9-8601fa7432a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:02.035393 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f73a36-257d-46fb-b6e4-6da1b0967cc6 None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:02.036612 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81993fb-86d1-46be-8fa9-8601fa7432a4 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:02.036710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f73a36-257d-46fb-b6e4-6da1b0967cc6 None admin] RBAC: Authorizing `identity:get_project(project_id=7abea439f4624f6680796de4a97fc13b)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:02.038157 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f73a36-257d-46fb-b6e4-6da1b0967cc6 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:02.039891 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 127/248] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:30:01 2022] GET /identity/v3/projects/7abea439f4624f6680796de4a97fc13b => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:02.041165 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af81544-a410-44cf-8424-49315c59d91d None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:02.041453 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af81544-a410-44cf-8424-49315c59d91d None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:02.041733 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af81544-a410-44cf-8424-49315c59d91d None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:02.043135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 128/249] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:02.046504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af81544-a410-44cf-8424-49315c59d91d None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:02.049339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 122/250] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:30:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:02.050872 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd4de9-5a20-4275-9f2c-f6a61afc5dd1 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:02.051378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd4de9-5a20-4275-9f2c-f6a61afc5dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:02.052045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd4de9-5a20-4275-9f2c-f6a61afc5dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:02.067163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:02.072821 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-eefd4de9-5a20-4275-9f2c-f6a61afc5dd1 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:02.074669 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-eefd4de9-5a20-4275-9f2c-f6a61afc5dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:02.161023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 123/251] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:02.161954 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd4de9-5a20-4275-9f2c-f6a61afc5dd1 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:02.184876 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:02.197808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af81544-a410-44cf-8424-49315c59d91d None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:02.253480 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a6fc7-dc50-4bd8-a612-3cc158337401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:02.254316 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a6fc7-dc50-4bd8-a612-3cc158337401 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:02.254604 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a6fc7-dc50-4bd8-a612-3cc158337401 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:02.254884 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a6fc7-dc50-4bd8-a612-3cc158337401 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:02.259503 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a6fc7-dc50-4bd8-a612-3cc158337401 None admin] RBAC: Adding query filter params (role.id=9170e7d540b446f3b5507688ca335ecc, scope.project.id=7abea439f4624f6680796de4a97fc13b, user.id=ea4d03e2ac6c4426951e5cfee25db55d) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:02.259841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a6fc7-dc50-4bd8-a612-3cc158337401 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:02.261964 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a6fc7-dc50-4bd8-a612-3cc158337401 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:02.272970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 129/252] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:30:02 2022] GET /identity/v3/role_assignments?user.id=ea4d03e2ac6c4426951e5cfee25db55d&scope.project.id=7abea439f4624f6680796de4a97fc13b&role.id=9170e7d540b446f3b5507688ca335ecc => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:02.273814 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a6fc7-dc50-4bd8-a612-3cc158337401 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:02.320010 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd4de9-5a20-4275-9f2c-f6a61afc5dd1 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:02.384240 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae02796-c9ba-463e-b0b4-0db3a8229a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:02.385173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae02796-c9ba-463e-b0b4-0db3a8229a25 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:02.385453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae02796-c9ba-463e-b0b4-0db3a8229a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:02.385726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae02796-c9ba-463e-b0b4-0db3a8229a25 None admin] PATH_INFO: `/v3/users/b591c0899b2f426e92cb2b7c82cb80a7` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:02.387733 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae02796-c9ba-463e-b0b4-0db3a8229a25 None admin] RBAC: Authorizing `identity:get_user(user_id=b591c0899b2f426e92cb2b7c82cb80a7)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:02.389586 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae02796-c9ba-463e-b0b4-0db3a8229a25 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:02.391930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 124/253] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:30:02 2022] GET /identity/v3/users/b591c0899b2f426e92cb2b7c82cb80a7 => generated 297 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:02.397991 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae02796-c9ba-463e-b0b4-0db3a8229a25 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:02.480983 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:02.539607 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a6fc7-dc50-4bd8-a612-3cc158337401 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:02.599924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:02.618297 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701e56c-2c24-4adf-8fd1-67f2c6f56dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:02.619495 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701e56c-2c24-4adf-8fd1-67f2c6f56dce None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:02.620157 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701e56c-2c24-4adf-8fd1-67f2c6f56dce None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:02.621791 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5701e56c-2c24-4adf-8fd1-67f2c6f56dce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:02.631445 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae02796-c9ba-463e-b0b4-0db3a8229a25 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:02.646078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5701e56c-2c24-4adf-8fd1-67f2c6f56dce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:02.656629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-5701e56c-2c24-4adf-8fd1-67f2c6f56dce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:30:02.656629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:02.656629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:02.656629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:02.656629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:02.656629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:02.656629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:02.656629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:02.656629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:02.656629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:02.656629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:02.656629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:02.656629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:02.656629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:02.656629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:02.656629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:02.657620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:02.657620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:02.657620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:02.657620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:02.657620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:02.657620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:02.657620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:02.657620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:02.657620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:02.657620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:02.657620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:02.657620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:02.657620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:02.657620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:02.657620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:02.657620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:02.658463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:02.658463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:02.658463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:02.658463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:02.658463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:02.658463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:02.658463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:02.658463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:02.658463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:02.658463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:02.658463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:30:02.658463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:02.664909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 130/254] 10.222.0.13 () {56 vars in 1114 bytes} [Fri May 27 19:30:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:02.672010 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701e56c-2c24-4adf-8fd1-67f2c6f56dce None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:02.714200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667aba4b-046e-46e1-b70d-16646822419b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:02.715541 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667aba4b-046e-46e1-b70d-16646822419b None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:02.716339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667aba4b-046e-46e1-b70d-16646822419b None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:02.716839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667aba4b-046e-46e1-b70d-16646822419b None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:02.718588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667aba4b-046e-46e1-b70d-16646822419b None admin] RBAC: Authorizing `identity:get_project(project_id=7abea439f4624f6680796de4a97fc13b)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:02.720302 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667aba4b-046e-46e1-b70d-16646822419b None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:02.725085 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 125/255] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:30:02 2022] GET /identity/v3/projects/7abea439f4624f6680796de4a97fc13b => generated 302 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:02.730447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667aba4b-046e-46e1-b70d-16646822419b None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:02.818378 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5701e56c-2c24-4adf-8fd1-67f2c6f56dce None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:02.855257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf219ff-3175-4a76-b1a7-a0b2f9e391d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:02.856819 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf219ff-3175-4a76-b1a7-a0b2f9e391d3 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:02.857328 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf219ff-3175-4a76-b1a7-a0b2f9e391d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:02.858028 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf219ff-3175-4a76-b1a7-a0b2f9e391d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:02.858989 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf219ff-3175-4a76-b1a7-a0b2f9e391d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:02.859496 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf219ff-3175-4a76-b1a7-a0b2f9e391d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:02.861138 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf219ff-3175-4a76-b1a7-a0b2f9e391d3 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:02.870304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 131/256] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:30:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:02.872209 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29459a5f-d0e7-4b1f-bfbb-654f331f6458 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:02.872981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29459a5f-d0e7-4b1f-bfbb-654f331f6458 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:02.873483 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29459a5f-d0e7-4b1f-bfbb-654f331f6458 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:02.875967 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 132/257] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:02.877160 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29459a5f-d0e7-4b1f-bfbb-654f331f6458 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:02.880473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:02.930670 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667aba4b-046e-46e1-b70d-16646822419b None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:02.970601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b38b1-1ff0-450c-a2c4-d7c905a15153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:02.971886 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b38b1-1ff0-450c-a2c4-d7c905a15153 None admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:02.972366 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b38b1-1ff0-450c-a2c4-d7c905a15153 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:02.972880 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b38b1-1ff0-450c-a2c4-d7c905a15153 None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b/users/b591c0899b2f426e92cb2b7c82cb80a7/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:02.975178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b38b1-1ff0-450c-a2c4-d7c905a15153 None admin] RBAC: Authorizing `identity:create_grant(project_id=7abea439f4624f6680796de4a97fc13b, user_id=b591c0899b2f426e92cb2b7c82cb80a7, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:02.976839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b38b1-1ff0-450c-a2c4-d7c905a15153 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:03.029140 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29459a5f-d0e7-4b1f-bfbb-654f331f6458 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:03.063930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 126/258] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:30:02 2022] PUT /identity/v3/projects/7abea439f4624f6680796de4a97fc13b/users/b591c0899b2f426e92cb2b7c82cb80a7/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:30:03.065638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ee21d-f37d-4dc9-b489-d513cfd31e0c None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:03.066273 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ee21d-f37d-4dc9-b489-d513cfd31e0c None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:03.066770 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ee21d-f37d-4dc9-b489-d513cfd31e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:03.089053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c713b-de39-4f43-9576-cbde24177a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:03.093725 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c713b-de39-4f43-9576-cbde24177a5e None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:03.094502 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c713b-de39-4f43-9576-cbde24177a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:03.095843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c713b-de39-4f43-9576-cbde24177a5e None admin] PATH_INFO: `/v3/users/21fef53ebab34efcb73b2dd125d2c640` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:03.099063 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c713b-de39-4f43-9576-cbde24177a5e None admin] RBAC: Authorizing `identity:get_user(user_id=21fef53ebab34efcb73b2dd125d2c640)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:03.102894 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c713b-de39-4f43-9576-cbde24177a5e None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:03.106742 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-787ee21d-f37d-4dc9-b489-d513cfd31e0c None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:03.107230 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 133/259] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:30:02 2022] GET /identity/v3/users/21fef53ebab34efcb73b2dd125d2c640 => generated 283 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:03.108768 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-787ee21d-f37d-4dc9-b489-d513cfd31e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:03.111827 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c713b-de39-4f43-9576-cbde24177a5e None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:03.136171 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:03.190045 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:03.235093 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c713b-de39-4f43-9576-cbde24177a5e None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:03.246970 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 127/260] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:03.292159 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546079c2-0739-48ef-892d-e59f0585ad4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:03.293214 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546079c2-0739-48ef-892d-e59f0585ad4a None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:03.294357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546079c2-0739-48ef-892d-e59f0585ad4a None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:03.294947 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546079c2-0739-48ef-892d-e59f0585ad4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:03.326995 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ee21d-f37d-4dc9-b489-d513cfd31e0c None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:03.375281 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546079c2-0739-48ef-892d-e59f0585ad4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:03.377304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546079c2-0739-48ef-892d-e59f0585ad4a None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:03.447844 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:03.495018 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 134/261] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:30:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:30:03.501085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546079c2-0739-48ef-892d-e59f0585ad4a None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:03.534846 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787ee21d-f37d-4dc9-b489-d513cfd31e0c None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:03.601770 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f288f05c-fcd8-4615-ab6a-6cdb2275e103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:03.602594 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288f05c-fcd8-4615-ab6a-6cdb2275e103 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:03.602885 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288f05c-fcd8-4615-ab6a-6cdb2275e103 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:03.603165 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288f05c-fcd8-4615-ab6a-6cdb2275e103 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:03.615235 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546079c2-0739-48ef-892d-e59f0585ad4a None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:03.623356 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f288f05c-fcd8-4615-ab6a-6cdb2275e103 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:03.640223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-f288f05c-fcd8-4615-ab6a-6cdb2275e103 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:30:03.640223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:03.640223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:03.640223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:03.640223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:03.640223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:03.640223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:03.640223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:03.640223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:03.640223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:03.640223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:03.640223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:03.640223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:03.640223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:03.640223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:03.640223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:03.641222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:03.641222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:03.641222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:03.641222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:03.641222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:03.641222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:03.641222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:03.641222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:03.641222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:03.641222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:03.641222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:03.641222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:03.641222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:03.641222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:03.641222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:03.641222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:03.642017 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:03.642017 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:03.642017 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:03.642017 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:03.642017 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:03.642017 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:03.642017 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:03.642017 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:03.642017 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:03.642017 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:03.642017 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:30:03.642017 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:03.642017 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 128/262] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:30:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:03.649540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f288f05c-fcd8-4615-ab6a-6cdb2275e103 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:03.661644 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04336c-98c1-4341-a55d-463dfe7f4d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:03.662518 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04336c-98c1-4341-a55d-463dfe7f4d21 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:03.662926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04336c-98c1-4341-a55d-463dfe7f4d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:03.663205 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e04336c-98c1-4341-a55d-463dfe7f4d21 None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:03.664437 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e04336c-98c1-4341-a55d-463dfe7f4d21 None admin] RBAC: Authorizing `identity:get_project(project_id=7abea439f4624f6680796de4a97fc13b)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:03.666126 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e04336c-98c1-4341-a55d-463dfe7f4d21 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:03.667971 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 135/263] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:30:03 2022] GET /identity/v3/projects/7abea439f4624f6680796de4a97fc13b => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:03.686824 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04336c-98c1-4341-a55d-463dfe7f4d21 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:03.701327 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:03.742953 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:03.833526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f288f05c-fcd8-4615-ab6a-6cdb2275e103 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:03.888252 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ecd4b-aea3-4887-a420-420fa0b2fcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:03.888937 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ecd4b-aea3-4887-a420-420fa0b2fcdd None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:03.889241 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ecd4b-aea3-4887-a420-420fa0b2fcdd None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:03.889516 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ecd4b-aea3-4887-a420-420fa0b2fcdd None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:03.890793 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ecd4b-aea3-4887-a420-420fa0b2fcdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:03.891045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ecd4b-aea3-4887-a420-420fa0b2fcdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:03.892436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ecd4b-aea3-4887-a420-420fa0b2fcdd None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:03.911377 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 129/264] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:03.917243 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ecd4b-aea3-4887-a420-420fa0b2fcdd None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:03.951660 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e04336c-98c1-4341-a55d-463dfe7f4d21 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:04.033313 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a6047-c9b4-4de5-b2c9-03527a2405fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:04.034351 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a6047-c9b4-4de5-b2c9-03527a2405fa None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:04.034954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a6047-c9b4-4de5-b2c9-03527a2405fa None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:04.035544 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162a6047-c9b4-4de5-b2c9-03527a2405fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:04.036578 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a6047-c9b4-4de5-b2c9-03527a2405fa None admin] RBAC: Adding query filter params (role.id=eda4ac05bc364224a7f319e7ded54073, scope.project.id=7abea439f4624f6680796de4a97fc13b, user.id=21fef53ebab34efcb73b2dd125d2c640) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:04.037130 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a6047-c9b4-4de5-b2c9-03527a2405fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:04.039002 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162a6047-c9b4-4de5-b2c9-03527a2405fa None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:04.052653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:04.053290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 136/265] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:30:03 2022] GET /identity/v3/role_assignments?user.id=21fef53ebab34efcb73b2dd125d2c640&scope.project.id=7abea439f4624f6680796de4a97fc13b&role.id=eda4ac05bc364224a7f319e7ded54073 => generated 608 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:04.107239 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ecd4b-aea3-4887-a420-420fa0b2fcdd None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:04.157311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9902b7-8ab8-4035-b872-7677b8a351f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:04.158138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9902b7-8ab8-4035-b872-7677b8a351f4 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:04.158427 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9902b7-8ab8-4035-b872-7677b8a351f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:04.158710 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9902b7-8ab8-4035-b872-7677b8a351f4 None admin] PATH_INFO: `/v3/users/21fef53ebab34efcb73b2dd125d2c640` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:04.160019 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9902b7-8ab8-4035-b872-7677b8a351f4 None admin] RBAC: Authorizing `identity:get_user(user_id=21fef53ebab34efcb73b2dd125d2c640)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:04.161514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9902b7-8ab8-4035-b872-7677b8a351f4 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:04.163467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 130/266] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:30:03 2022] GET /identity/v3/users/21fef53ebab34efcb73b2dd125d2c640 => generated 283 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:04.170037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a6047-c9b4-4de5-b2c9-03527a2405fa None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:04.332497 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:04.339315 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162a6047-c9b4-4de5-b2c9-03527a2405fa None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:04.390846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ae13c-c548-4dd8-a2eb-e58b0682a831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:04.391541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ae13c-c548-4dd8-a2eb-e58b0682a831 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:04.391827 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ae13c-c548-4dd8-a2eb-e58b0682a831 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:04.392106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468ae13c-c548-4dd8-a2eb-e58b0682a831 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:04.474902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ae13c-c548-4dd8-a2eb-e58b0682a831 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:04.476856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468ae13c-c548-4dd8-a2eb-e58b0682a831 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:04.579615 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:04.587472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 137/267] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:30:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:30:04.594454 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9902b7-8ab8-4035-b872-7677b8a351f4 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:04.722412 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9902b7-8ab8-4035-b872-7677b8a351f4 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:04.768191 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0eeb1-4128-4125-88c7-7524c6d7d76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:04.769000 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0eeb1-4128-4125-88c7-7524c6d7d76a None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:04.769285 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0eeb1-4128-4125-88c7-7524c6d7d76a None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:04.769556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0eeb1-4128-4125-88c7-7524c6d7d76a None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:04.770744 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0eeb1-4128-4125-88c7-7524c6d7d76a None admin] RBAC: Authorizing `identity:get_project(project_id=7abea439f4624f6680796de4a97fc13b)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:04.772210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0eeb1-4128-4125-88c7-7524c6d7d76a None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:04.774060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 131/268] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:30:04 2022] GET /identity/v3/projects/7abea439f4624f6680796de4a97fc13b => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:04.780691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ae13c-c548-4dd8-a2eb-e58b0682a831 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:04.916739 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468ae13c-c548-4dd8-a2eb-e58b0682a831 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:04.960466 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8faaa-9b4a-4d3e-a926-96cefa8f1f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:04.961174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8faaa-9b4a-4d3e-a926-96cefa8f1f69 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:04.961465 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8faaa-9b4a-4d3e-a926-96cefa8f1f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:04.961744 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8faaa-9b4a-4d3e-a926-96cefa8f1f69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:04.962586 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8faaa-9b4a-4d3e-a926-96cefa8f1f69 None admin] RBAC: Adding query filter params (role.id=6505bbb129404436966ee59aa7a01d8e, scope.project.id=7abea439f4624f6680796de4a97fc13b, user.id=21fef53ebab34efcb73b2dd125d2c640) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:04.962904 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8faaa-9b4a-4d3e-a926-96cefa8f1f69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:04.964376 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8faaa-9b4a-4d3e-a926-96cefa8f1f69 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:04.971970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 138/269] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:30:04 2022] GET /identity/v3/role_assignments?user.id=21fef53ebab34efcb73b2dd125d2c640&scope.project.id=7abea439f4624f6680796de4a97fc13b&role.id=6505bbb129404436966ee59aa7a01d8e => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:05.040642 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfae3f-9c35-40e2-86f2-1159b8747071 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:05.040826 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfae3f-9c35-40e2-86f2-1159b8747071 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:05.041132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfae3f-9c35-40e2-86f2-1159b8747071 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:05.042719 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 132/270] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:05.049558 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b31a73-5ae2-4b8b-aa09-1f5f333b7ff4 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:05.050209 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b31a73-5ae2-4b8b-aa09-1f5f333b7ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:05.050703 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b31a73-5ae2-4b8b-aa09-1f5f333b7ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:05.072569 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-23b31a73-5ae2-4b8b-aa09-1f5f333b7ff4 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:05.074199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-23b31a73-5ae2-4b8b-aa09-1f5f333b7ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:05.168650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:05.181481 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 139/271] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:05.197354 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a749210-5ab4-4f38-9922-bd22dbbe2ba5 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:05.197644 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a749210-5ab4-4f38-9922-bd22dbbe2ba5 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:05.197934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a749210-5ab4-4f38-9922-bd22dbbe2ba5 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:05.199837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 133/272] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:05.205775 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95049b-cbf6-4d29-8fcd-e5a9ec5205d0 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:05.206076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95049b-cbf6-4d29-8fcd-e5a9ec5205d0 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:05.206360 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b95049b-cbf6-4d29-8fcd-e5a9ec5205d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:05.221298 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a749210-5ab4-4f38-9922-bd22dbbe2ba5 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:05.225503 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:05.233723 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-5b95049b-cbf6-4d29-8fcd-e5a9ec5205d0 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:05.235576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-5b95049b-cbf6-4d29-8fcd-e5a9ec5205d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:05.357983 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 140/273] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:05.359304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250e87d-503a-4fce-ad90-f196b625a464 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:05.359703 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250e87d-503a-4fce-ad90-f196b625a464 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:05.360076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1250e87d-503a-4fce-ad90-f196b625a464 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:05.365365 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 141/274] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:05.368969 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc375fc-f901-4478-8486-64a4f2215256 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:05.368969 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc375fc-f901-4478-8486-64a4f2215256 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:05.368969 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc375fc-f901-4478-8486-64a4f2215256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:05.389702 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a749210-5ab4-4f38-9922-bd22dbbe2ba5 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:05.390237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-dfc375fc-f901-4478-8486-64a4f2215256 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:05.391750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-dfc375fc-f901-4478-8486-64a4f2215256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:05.430876 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed98a9-ce05-4d40-a704-54b3264d0b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:05.431687 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed98a9-ce05-4d40-a704-54b3264d0b9f None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:05.432453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed98a9-ce05-4d40-a704-54b3264d0b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:05.432453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed98a9-ce05-4d40-a704-54b3264d0b9f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:05.434778 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ed98a9-ce05-4d40-a704-54b3264d0b9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:05.436539 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ed98a9-ce05-4d40-a704-54b3264d0b9f None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:05.438444 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 134/275] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:05.439051 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed98a9-ce05-4d40-a704-54b3264d0b9f None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:05.483421 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 142/276] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 19:30:05.484289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc375fc-f901-4478-8486-64a4f2215256 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:05.524582 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:05.556150 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:05.557035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed98a9-ce05-4d40-a704-54b3264d0b9f None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:05.593970 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b845e-5d32-4845-a555-635516361cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:05.594762 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b845e-5d32-4845-a555-635516361cdd None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:05.595051 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b845e-5d32-4845-a555-635516361cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:05.595326 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b845e-5d32-4845-a555-635516361cdd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:05.607114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b845e-5d32-4845-a555-635516361cdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:05.613527 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc375fc-f901-4478-8486-64a4f2215256 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:05.614829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-ee0b845e-5d32-4845-a555-635516361cdd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 19:30:05.614829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:05.614829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:05.614829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:05.614829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:05.614829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:05.614829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:05.614829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:05.614829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:05.614829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:05.614829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:05.614829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:05.614829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:05.614829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:05.614829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:05.614829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:05.615800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:05.615800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:05.615800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:05.615800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:05.615800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:05.615800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:05.615800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:05.615800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:05.615800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:05.615800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:05.615800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:05.615800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:05.615800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:05.615800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:05.615800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:05.615800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:05.617981 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:05.617981 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:05.617981 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:05.617981 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:05.617981 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:05.617981 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:05.617981 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:05.617981 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:05.617981 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:05.617981 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:05.617981 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 19:30:05.617981 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:05.617981 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 135/277] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:05.617981 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b845e-5d32-4845-a555-635516361cdd None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:05.658566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef0132-c6b9-46b1-a205-0788688cdc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:05.662052 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef0132-c6b9-46b1-a205-0788688cdc9c None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:05.662052 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef0132-c6b9-46b1-a205-0788688cdc9c None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:05.662052 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef0132-c6b9-46b1-a205-0788688cdc9c None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:05.662052 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef0132-c6b9-46b1-a205-0788688cdc9c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:05.662761 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef0132-c6b9-46b1-a205-0788688cdc9c None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:05.723909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b845e-5d32-4845-a555-635516361cdd None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:05.753240 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:05.758072 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9548e8f-3ba0-4219-8298-5add4426481f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:05.758870 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9548e8f-3ba0-4219-8298-5add4426481f None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:05.759153 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9548e8f-3ba0-4219-8298-5add4426481f None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:05.759466 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9548e8f-3ba0-4219-8298-5add4426481f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:05.765499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 143/278] 10.222.0.13 () {60 vars in 1132 bytes} [Fri May 27 19:30:05 2022] POST /identity/v3/users => generated 293 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:30:05.766507 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef0132-c6b9-46b1-a205-0788688cdc9c None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:05.782578 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9548e8f-3ba0-4219-8298-5add4426481f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:05.794136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-a9548e8f-3ba0-4219-8298-5add4426481f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 19:30:05.794136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:05.794136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:05.794136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:05.794136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:05.794136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:05.794136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:05.794136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:05.794136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:05.794136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:05.794136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:05.794136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:05.794136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:05.794136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:05.794136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:05.794136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:05.795119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:05.795119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:05.795119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:05.795119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:05.795119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:05.795119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:05.795119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:05.795119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:05.795119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:05.795119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:05.795119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:05.795119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:05.795119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:05.795119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:05.795119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:05.795119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:05.796115 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:05.796115 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:05.796115 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:05.796115 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:05.796115 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:05.796115 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:05.796115 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:05.796115 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:05.796115 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:05.796115 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:05.796115 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 19:30:05.796115 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:05.804721 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 136/279] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:05.806050 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9548e8f-3ba0-4219-8298-5add4426481f None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:05.917501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef0132-c6b9-46b1-a205-0788688cdc9c None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:05.925933 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:05.957023 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485ac93-29a0-4b69-9859-eb707ff51510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:05.957738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485ac93-29a0-4b69-9859-eb707ff51510 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:05.958025 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485ac93-29a0-4b69-9859-eb707ff51510 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:05.958526 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485ac93-29a0-4b69-9859-eb707ff51510 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:05.959104 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485ac93-29a0-4b69-9859-eb707ff51510 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:05.959426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485ac93-29a0-4b69-9859-eb707ff51510 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:05.960849 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485ac93-29a0-4b69-9859-eb707ff51510 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:05.962941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9548e8f-3ba0-4219-8298-5add4426481f None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:05.974136 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 144/280] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:30:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:05.979994 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485ac93-29a0-4b69-9859-eb707ff51510 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:06.014222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3803987-884c-4e0b-beb2-c1e9290ff5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:06.015792 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3803987-884c-4e0b-beb2-c1e9290ff5a5 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:06.015792 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3803987-884c-4e0b-beb2-c1e9290ff5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:06.015792 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3803987-884c-4e0b-beb2-c1e9290ff5a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:06.016214 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3803987-884c-4e0b-beb2-c1e9290ff5a5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:06.016526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3803987-884c-4e0b-beb2-c1e9290ff5a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:06.018022 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3803987-884c-4e0b-beb2-c1e9290ff5a5 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:06.033046 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 137/281] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:30:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:06.034932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3803987-884c-4e0b-beb2-c1e9290ff5a5 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:06.102590 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:06.106787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485ac93-29a0-4b69-9859-eb707ff51510 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:06.157584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b16f53-39e0-470d-a9c6-5a2d78c5b8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:06.159090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b16f53-39e0-470d-a9c6-5a2d78c5b8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:06.159400 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b16f53-39e0-470d-a9c6-5a2d78c5b8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:06.159683 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b16f53-39e0-470d-a9c6-5a2d78c5b8c6 None admin] PATH_INFO: `/v3/users/ea4d03e2ac6c4426951e5cfee25db55d` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:06.160843 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3803987-884c-4e0b-beb2-c1e9290ff5a5 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:06.161384 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b16f53-39e0-470d-a9c6-5a2d78c5b8c6 None admin] RBAC: Authorizing `identity:get_user(user_id=ea4d03e2ac6c4426951e5cfee25db55d)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:06.162856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b16f53-39e0-470d-a9c6-5a2d78c5b8c6 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:06.164499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 145/282] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:30:05 2022] GET /identity/v3/users/ea4d03e2ac6c4426951e5cfee25db55d => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:06.171108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b16f53-39e0-470d-a9c6-5a2d78c5b8c6 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:06.208991 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3addd494-5e8d-4837-98d5-5f9d5e58eefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:06.209894 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3addd494-5e8d-4837-98d5-5f9d5e58eefc None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:06.210184 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3addd494-5e8d-4837-98d5-5f9d5e58eefc None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:06.210466 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3addd494-5e8d-4837-98d5-5f9d5e58eefc None admin] PATH_INFO: `/v3/users/3bb65f7f038e40d993084aec66fdfbb9` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:06.221897 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:06.230052 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3addd494-5e8d-4837-98d5-5f9d5e58eefc None admin] RBAC: Authorizing `identity:get_user(user_id=3bb65f7f038e40d993084aec66fdfbb9)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:06.231823 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3addd494-5e8d-4837-98d5-5f9d5e58eefc None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:06.234117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 138/283] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:30:06 2022] GET /identity/v3/users/3bb65f7f038e40d993084aec66fdfbb9 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:06.239638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3addd494-5e8d-4837-98d5-5f9d5e58eefc None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:06.299927 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b16f53-39e0-470d-a9c6-5a2d78c5b8c6 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:06.354969 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3bc0f-7913-4c9e-8d41-6cd20b7fe86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:06.356691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3bc0f-7913-4c9e-8d41-6cd20b7fe86c None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:06.356691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3bc0f-7913-4c9e-8d41-6cd20b7fe86c None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:06.357090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3bc0f-7913-4c9e-8d41-6cd20b7fe86c None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:06.358435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c3bc0f-7913-4c9e-8d41-6cd20b7fe86c None admin] RBAC: Authorizing `identity:get_project(project_id=7abea439f4624f6680796de4a97fc13b)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:06.359800 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c3bc0f-7913-4c9e-8d41-6cd20b7fe86c None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:06.361835 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 146/284] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:30:06 2022] GET /identity/v3/projects/7abea439f4624f6680796de4a97fc13b => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:06.363045 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2d62c-c5a1-4a89-bd6d-57adb7ee5372 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:06.363326 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2d62c-c5a1-4a89-bd6d-57adb7ee5372 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:06.363602 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2d62c-c5a1-4a89-bd6d-57adb7ee5372 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:06.364818 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 147/285] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:06.370530 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a56ac-eb6a-4e5c-808d-2a37472da418 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:06.370816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a56ac-eb6a-4e5c-808d-2a37472da418 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:06.371094 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a56ac-eb6a-4e5c-808d-2a37472da418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:06.372421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3addd494-5e8d-4837-98d5-5f9d5e58eefc None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:06.389992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:06.391000 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-d94a56ac-eb6a-4e5c-808d-2a37472da418 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:06.393079 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d94a56ac-eb6a-4e5c-808d-2a37472da418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:06.418786 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e53cd-feea-47da-887b-52341320ab75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:06.420570 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e53cd-feea-47da-887b-52341320ab75 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:06.421321 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e53cd-feea-47da-887b-52341320ab75 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:06.421935 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e53cd-feea-47da-887b-52341320ab75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:06.423253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e53cd-feea-47da-887b-52341320ab75 None admin] RBAC: Adding query filter params (role.id=cb43a575e47147c799eec2f362f4fe0e, user.id=3bb65f7f038e40d993084aec66fdfbb9, scope.system=all) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:06.424133 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e53cd-feea-47da-887b-52341320ab75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:06.426411 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e53cd-feea-47da-887b-52341320ab75 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:06.435582 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 139/286] 10.222.0.13 () {56 vars in 1308 bytes} [Fri May 27 19:30:06 2022] GET /identity/v3/role_assignments?user.id=3bb65f7f038e40d993084aec66fdfbb9&scope.system=all&role.id=cb43a575e47147c799eec2f362f4fe0e => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:06.436718 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e53cd-feea-47da-887b-52341320ab75 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:06.473795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 148/287] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:06.499110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a56ac-eb6a-4e5c-808d-2a37472da418 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:06.550859 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:06.578478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e53cd-feea-47da-887b-52341320ab75 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:06.637508 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a56ac-eb6a-4e5c-808d-2a37472da418 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:06.667022 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee6f92-cb69-4ec9-966a-3a2ab6606906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:06.668422 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee6f92-cb69-4ec9-966a-3a2ab6606906 None admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:06.668942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee6f92-cb69-4ec9-966a-3a2ab6606906 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:06.669375 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee6f92-cb69-4ec9-966a-3a2ab6606906 None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b/users/ea4d03e2ac6c4426951e5cfee25db55d/roles/9170e7d540b446f3b5507688ca335ecc` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:06.673184 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee6f92-cb69-4ec9-966a-3a2ab6606906 None admin] RBAC: Authorizing `identity:create_grant(project_id=7abea439f4624f6680796de4a97fc13b, user_id=ea4d03e2ac6c4426951e5cfee25db55d, role_id=9170e7d540b446f3b5507688ca335ecc)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:06.674908 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee6f92-cb69-4ec9-966a-3a2ab6606906 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:06.680349 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8da08-8540-4556-ae10-719d3c7f0522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:06.681317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8da08-8540-4556-ae10-719d3c7f0522 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:06.681724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8da08-8540-4556-ae10-719d3c7f0522 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:06.682234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8da08-8540-4556-ae10-719d3c7f0522 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:06.699630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a8da08-8540-4556-ae10-719d3c7f0522 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:06.708705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-83a8da08-8540-4556-ae10-719d3c7f0522 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 19:30:06.708705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:06.708705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:06.708705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:06.708705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:06.708705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:06.708705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:06.708705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:06.708705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:06.708705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:06.708705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:06.708705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:06.708705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:06.708705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:06.708705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:06.708705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:06.709710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:06.709710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:06.709710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:06.709710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:06.709710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:06.709710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:06.709710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:06.709710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:06.709710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:06.709710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:06.709710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:06.709710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:06.709710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:06.709710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:06.709710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:06.709710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:06.710494 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:06.710494 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:06.710494 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:06.710494 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:06.710494 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:06.710494 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:06.710494 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:06.710494 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:06.710494 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:06.710494 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:06.710494 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 19:30:06.710494 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:06.712798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 149/288] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:06.720355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8da08-8540-4556-ae10-719d3c7f0522 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:06.756376 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 140/289] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:30:06 2022] PUT /identity/v3/projects/7abea439f4624f6680796de4a97fc13b/users/ea4d03e2ac6c4426951e5cfee25db55d/roles/9170e7d540b446f3b5507688ca335ecc => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:30:06.843199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:06.850637 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8da08-8540-4556-ae10-719d3c7f0522 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:06.903431 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad26279-91bb-433c-8c40-fbcfb94e60ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:06.904554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad26279-91bb-433c-8c40-fbcfb94e60ed None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:06.905118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad26279-91bb-433c-8c40-fbcfb94e60ed None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:06.905606 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad26279-91bb-433c-8c40-fbcfb94e60ed None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:06.906606 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad26279-91bb-433c-8c40-fbcfb94e60ed None admin] RBAC: Adding query filter params (name=member) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:06.907119 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad26279-91bb-433c-8c40-fbcfb94e60ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:06.908792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad26279-91bb-433c-8c40-fbcfb94e60ed None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:06.925968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 150/290] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:30:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:06.932386 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee6f92-cb69-4ec9-966a-3a2ab6606906 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:06.973627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:07.045814 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee6f92-cb69-4ec9-966a-3a2ab6606906 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:07.088009 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55694e93-e6b8-404a-b7aa-51a3439db034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:07.089424 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55694e93-e6b8-404a-b7aa-51a3439db034 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:07.089956 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55694e93-e6b8-404a-b7aa-51a3439db034 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:07.090657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55694e93-e6b8-404a-b7aa-51a3439db034 None admin] PATH_INFO: `/v3/users/b591c0899b2f426e92cb2b7c82cb80a7` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:07.092001 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55694e93-e6b8-404a-b7aa-51a3439db034 None admin] RBAC: Authorizing `identity:get_user(user_id=b591c0899b2f426e92cb2b7c82cb80a7)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:07.093496 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55694e93-e6b8-404a-b7aa-51a3439db034 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:07.095368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 141/291] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:30:06 2022] GET /identity/v3/users/b591c0899b2f426e92cb2b7c82cb80a7 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:07.100671 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad26279-91bb-433c-8c40-fbcfb94e60ed None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:07.211237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad26279-91bb-433c-8c40-fbcfb94e60ed None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:07.252105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b5f07-2c1e-4ab8-927b-f2cf880cf1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:07.252773 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b5f07-2c1e-4ab8-927b-f2cf880cf1e0 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:07.253078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b5f07-2c1e-4ab8-927b-f2cf880cf1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:07.253393 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b5f07-2c1e-4ab8-927b-f2cf880cf1e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:07.287148 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:07.350074 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b5f07-2c1e-4ab8-927b-f2cf880cf1e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:07.351551 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b5f07-2c1e-4ab8-927b-f2cf880cf1e0 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:07.478667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 151/292] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:30:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:30:07.485005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55694e93-e6b8-404a-b7aa-51a3439db034 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:07.646088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55694e93-e6b8-404a-b7aa-51a3439db034 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:07.654599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:07.698762 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb14285-aa33-4b22-a857-4855905d71ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:07.700182 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb14285-aa33-4b22-a857-4855905d71ef None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:07.700679 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb14285-aa33-4b22-a857-4855905d71ef None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:07.701184 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb14285-aa33-4b22-a857-4855905d71ef None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:07.703843 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb14285-aa33-4b22-a857-4855905d71ef None admin] RBAC: Authorizing `identity:get_project(project_id=7abea439f4624f6680796de4a97fc13b)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:07.705557 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb14285-aa33-4b22-a857-4855905d71ef None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:07.707497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 142/293] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:30:07 2022] GET /identity/v3/projects/7abea439f4624f6680796de4a97fc13b => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:07.714381 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b5f07-2c1e-4ab8-927b-f2cf880cf1e0 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:07.872408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:07.927283 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b5f07-2c1e-4ab8-927b-f2cf880cf1e0 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:07.978759 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84729da-a39a-4145-9e94-09d5433527a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:07.979444 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84729da-a39a-4145-9e94-09d5433527a6 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:07.979732 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84729da-a39a-4145-9e94-09d5433527a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:07.980009 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84729da-a39a-4145-9e94-09d5433527a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:07.981500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84729da-a39a-4145-9e94-09d5433527a6 None admin] RBAC: Adding query filter params (role.id=cb43a575e47147c799eec2f362f4fe0e, scope.project.id=7abea439f4624f6680796de4a97fc13b, user.id=b591c0899b2f426e92cb2b7c82cb80a7) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:07.981814 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84729da-a39a-4145-9e94-09d5433527a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:07.983279 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84729da-a39a-4145-9e94-09d5433527a6 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:07.994409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 152/294] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:30:07 2022] GET /identity/v3/role_assignments?user.id=b591c0899b2f426e92cb2b7c82cb80a7&scope.project.id=7abea439f4624f6680796de4a97fc13b&role.id=cb43a575e47147c799eec2f362f4fe0e => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:08.451125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f312b-a668-4fe2-9128-73143bbd68fe None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:08.451641 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f312b-a668-4fe2-9128-73143bbd68fe None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:08.451718 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f312b-a668-4fe2-9128-73143bbd68fe None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:08.453352 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 143/295] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:08.460002 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c49fc-6a47-43c7-a974-c98e3b28200f None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:08.460294 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c49fc-6a47-43c7-a974-c98e3b28200f None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:08.460576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c49fc-6a47-43c7-a974-c98e3b28200f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:08.478576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-b32c49fc-6a47-43c7-a974-c98e3b28200f None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:08.479756 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-b32c49fc-6a47-43c7-a974-c98e3b28200f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:08.522063 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e009bb3-0cc4-4c8c-883a-5df013f8bced None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:08.522693 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e009bb3-0cc4-4c8c-883a-5df013f8bced None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:08.523177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e009bb3-0cc4-4c8c-883a-5df013f8bced None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:08.524793 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 144/296] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:08.531717 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffef2a-87cf-4c75-84a8-583e5519abe1 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:08.532302 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffef2a-87cf-4c75-84a8-583e5519abe1 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:08.532783 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffef2a-87cf-4c75-84a8-583e5519abe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:08.553854 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-58ffef2a-87cf-4c75-84a8-583e5519abe1 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:08.555869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-58ffef2a-87cf-4c75-84a8-583e5519abe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:08.575716 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 153/297] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:08.607784 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c49fc-6a47-43c7-a974-c98e3b28200f None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:08.651167 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 145/298] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 19:30:08.677823 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ffef2a-87cf-4c75-84a8-583e5519abe1 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:08.678178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:08.740049 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c49fc-6a47-43c7-a974-c98e3b28200f None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:08.778014 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cba2d-bf2a-498e-9978-02e526ab82a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:08.779007 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cba2d-bf2a-498e-9978-02e526ab82a4 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:08.779353 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cba2d-bf2a-498e-9978-02e526ab82a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:08.779699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069cba2d-bf2a-498e-9978-02e526ab82a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:08.781016 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cba2d-bf2a-498e-9978-02e526ab82a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:08.782565 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069cba2d-bf2a-498e-9978-02e526ab82a4 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:08.784255 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 154/299] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:08.789977 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cba2d-bf2a-498e-9978-02e526ab82a4 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:08.818626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ffef2a-87cf-4c75-84a8-583e5519abe1 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:08.820399 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:08.858484 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304bf6a-38d2-4f35-8582-161f168a3938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:08.859515 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304bf6a-38d2-4f35-8582-161f168a3938 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:08.859870 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304bf6a-38d2-4f35-8582-161f168a3938 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:08.860316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304bf6a-38d2-4f35-8582-161f168a3938 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:08.874321 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8304bf6a-38d2-4f35-8582-161f168a3938 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:08.883230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-8304bf6a-38d2-4f35-8582-161f168a3938 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 19:30:08.883230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:08.883230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:08.883230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:08.883230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:08.883230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:08.883230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:08.883230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:08.883230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:08.883230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:08.883230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:08.883230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:08.883230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:08.883230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:08.883230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:08.883230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:08.884151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:08.884151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:08.884151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:08.884151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:08.884151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:08.884151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:08.884151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:08.884151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:08.884151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:08.884151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:08.884151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:08.884151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:08.884151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:08.884151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:08.884151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:08.884151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:08.885008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:08.885008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:08.885008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:08.885008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:08.885008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:08.885008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:08.885008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:08.885008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:08.885008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:08.885008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:08.885008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 19:30:08.885008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:08.885008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 146/300] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:30:08.892278 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304bf6a-38d2-4f35-8582-161f168a3938 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:08.928923 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069cba2d-bf2a-498e-9978-02e526ab82a4 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:08.955918 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:08.979253 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510daf6-706b-4ff9-b678-271ee98bbbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:08.980936 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510daf6-706b-4ff9-b678-271ee98bbbb6 None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:08.981589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510daf6-706b-4ff9-b678-271ee98bbbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:08.982180 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c510daf6-706b-4ff9-b678-271ee98bbbb6 None admin] PATH_INFO: `/v3/groups` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:08.983152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510daf6-706b-4ff9-b678-271ee98bbbb6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:08.985453 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c510daf6-706b-4ff9-b678-271ee98bbbb6 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:09.035276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304bf6a-38d2-4f35-8582-161f168a3938 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:09.066902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 155/301] 10.222.0.13 () {60 vars in 1134 bytes} [Fri May 27 19:30:08 2022] POST /identity/v3/groups => generated 229 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:30:09.091170 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a5dba-af37-4c3f-bf56-90facda0cdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:09.092038 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a5dba-af37-4c3f-bf56-90facda0cdd7 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:09.092331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a5dba-af37-4c3f-bf56-90facda0cdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:09.092617 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091a5dba-af37-4c3f-bf56-90facda0cdd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:09.093432 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a5dba-af37-4c3f-bf56-90facda0cdd7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:09.093747 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a5dba-af37-4c3f-bf56-90facda0cdd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:09.095228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091a5dba-af37-4c3f-bf56-90facda0cdd7 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:09.111009 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 147/302] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:30:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:09.113041 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510daf6-706b-4ff9-b678-271ee98bbbb6 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:09.235380 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c510daf6-706b-4ff9-b678-271ee98bbbb6 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:09.276910 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77df7eba-6838-45a7-aa61-59bb89883346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:09.278318 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df7eba-6838-45a7-aa61-59bb89883346 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:09.278812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df7eba-6838-45a7-aa61-59bb89883346 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:09.279291 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77df7eba-6838-45a7-aa61-59bb89883346 None admin] PATH_INFO: `/v3/users/33d779b5c9fd495e8379f3d1f838e6ce` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:09.295580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df7eba-6838-45a7-aa61-59bb89883346 None admin] RBAC: Authorizing `identity:get_user(user_id=33d779b5c9fd495e8379f3d1f838e6ce)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:09.297430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77df7eba-6838-45a7-aa61-59bb89883346 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:09.299732 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 156/303] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:30:09 2022] GET /identity/v3/users/33d779b5c9fd495e8379f3d1f838e6ce => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:09.305724 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a5dba-af37-4c3f-bf56-90facda0cdd7 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:09.336262 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:09.420595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e820cd3-a687-4a1f-902b-466980c2bf18 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:09.420927 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e820cd3-a687-4a1f-902b-466980c2bf18 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:09.421195 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e820cd3-a687-4a1f-902b-466980c2bf18 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:09.422932 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 157/304] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 19:30:09.423088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091a5dba-af37-4c3f-bf56-90facda0cdd7 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:09.427976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc4d3e-c712-4ad4-ad9a-3371c2723e3a None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:09.428266 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc4d3e-c712-4ad4-ad9a-3371c2723e3a None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:09.428655 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc4d3e-c712-4ad4-ad9a-3371c2723e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:09.446953 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-b0cc4d3e-c712-4ad4-ad9a-3371c2723e3a None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:09.448248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-b0cc4d3e-c712-4ad4-ad9a-3371c2723e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:09.467648 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b830d67-6b3a-4ce4-b611-3e505abc2306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:09.469299 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b830d67-6b3a-4ce4-b611-3e505abc2306 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:09.469830 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b830d67-6b3a-4ce4-b611-3e505abc2306 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:09.470252 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b830d67-6b3a-4ce4-b611-3e505abc2306 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:09.471235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b830d67-6b3a-4ce4-b611-3e505abc2306 None admin] RBAC: Adding query filter params (role.id=9170e7d540b446f3b5507688ca335ecc, user.id=33d779b5c9fd495e8379f3d1f838e6ce, scope.system=all) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:09.471693 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b830d67-6b3a-4ce4-b611-3e505abc2306 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:09.473899 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b830d67-6b3a-4ce4-b611-3e505abc2306 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:09.484528 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 148/305] 10.222.0.13 () {56 vars in 1308 bytes} [Fri May 27 19:30:09 2022] GET /identity/v3/role_assignments?user.id=33d779b5c9fd495e8379f3d1f838e6ce&scope.system=all&role.id=9170e7d540b446f3b5507688ca335ecc => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:09.491087 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:09.535204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 158/306] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 19:30:09.559387 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b830d67-6b3a-4ce4-b611-3e505abc2306 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:09.677450 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:09.691106 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b830d67-6b3a-4ce4-b611-3e505abc2306 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:09.727964 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b2730-5fde-4567-98d2-de87d2ceb31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:09.728774 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b2730-5fde-4567-98d2-de87d2ceb31e None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:09.729097 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b2730-5fde-4567-98d2-de87d2ceb31e None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:09.729380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b2730-5fde-4567-98d2-de87d2ceb31e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:09.740941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b2730-5fde-4567-98d2-de87d2ceb31e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:09.749277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-c03b2730-5fde-4567-98d2-de87d2ceb31e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 19:30:09.749277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:09.749277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:09.749277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:09.749277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:09.749277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:09.749277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:09.749277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:09.749277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:09.749277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:09.749277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:09.749277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:09.749277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:09.749277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:09.749277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:09.749277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:09.750301 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:09.750301 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:09.750301 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:09.750301 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:09.750301 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:09.750301 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:09.750301 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:09.750301 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:09.750301 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:09.750301 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:09.750301 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:09.750301 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:09.750301 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:09.750301 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:09.750301 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:09.750301 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:09.751109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:09.751109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:09.751109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:09.751109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:09.751109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:09.751109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:09.751109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:09.751109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:09.751109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:09.751109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:09.751109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 19:30:09.751109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:09.751109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 149/307] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:09.757727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc4d3e-c712-4ad4-ad9a-3371c2723e3a None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:09.767536 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2146b89-129f-4335-812c-b164efa91298 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:09.767856 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2146b89-129f-4335-812c-b164efa91298 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:09.768101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2146b89-129f-4335-812c-b164efa91298 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:09.769755 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 150/308] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:09.775361 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d631b7e-eeac-4989-b195-ef3557ad92eb None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:09.775810 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d631b7e-eeac-4989-b195-ef3557ad92eb None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:09.776242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d631b7e-eeac-4989-b195-ef3557ad92eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:09.796533 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-0d631b7e-eeac-4989-b195-ef3557ad92eb None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:09.797903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-0d631b7e-eeac-4989-b195-ef3557ad92eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:09.879768 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 151/309] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:09.886274 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc4d3e-c712-4ad4-ad9a-3371c2723e3a None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:09.904284 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d631b7e-eeac-4989-b195-ef3557ad92eb None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:09.931184 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51548daf-b8f6-422c-b0bb-f83746f66280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:09.933068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51548daf-b8f6-422c-b0bb-f83746f66280 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:09.934070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51548daf-b8f6-422c-b0bb-f83746f66280 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:09.934631 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51548daf-b8f6-422c-b0bb-f83746f66280 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:09.935824 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51548daf-b8f6-422c-b0bb-f83746f66280 None admin] RBAC: Adding query filter params (name=member) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:09.936275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51548daf-b8f6-422c-b0bb-f83746f66280 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:09.938691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51548daf-b8f6-422c-b0bb-f83746f66280 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:09.951340 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 159/310] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:30:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:09.956338 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51548daf-b8f6-422c-b0bb-f83746f66280 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:09.996835 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:10.005897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:10.040929 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d631b7e-eeac-4989-b195-ef3557ad92eb None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:10.081768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51548daf-b8f6-422c-b0bb-f83746f66280 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:10.083226 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea4a7d-d129-43d8-81c3-e2e64ba4f330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:10.084087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea4a7d-d129-43d8-81c3-e2e64ba4f330 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:10.084311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea4a7d-d129-43d8-81c3-e2e64ba4f330 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:10.084585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ea4a7d-d129-43d8-81c3-e2e64ba4f330 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:10.098098 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ea4a7d-d129-43d8-81c3-e2e64ba4f330 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:10.106764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-d6ea4a7d-d129-43d8-81c3-e2e64ba4f330 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 19:30:10.106764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:10.106764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:10.106764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:10.106764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:10.106764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:10.106764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:10.106764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:10.106764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:10.106764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:10.106764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:10.106764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:10.106764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:10.106764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:10.106764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:10.106764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:10.107695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:10.107695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:10.107695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:10.107695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:10.107695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:10.107695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:10.107695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:10.107695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:10.107695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:10.107695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:10.107695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:10.107695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:10.107695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:10.107695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:10.107695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:10.107695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:10.108490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:10.108490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:10.108490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:10.108490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:10.108490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:10.108490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:10.108490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:10.108490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:10.108490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:10.108490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:10.108490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 19:30:10.108490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:10.112996 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 152/311] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:10.114360 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea4a7d-d129-43d8-81c3-e2e64ba4f330 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:10.133035 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e742e02-0fbb-45e0-9391-73eeaa10b963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:10.134425 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e742e02-0fbb-45e0-9391-73eeaa10b963 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:10.135109 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e742e02-0fbb-45e0-9391-73eeaa10b963 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:10.135604 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e742e02-0fbb-45e0-9391-73eeaa10b963 None admin] PATH_INFO: `/v3/users/3bb65f7f038e40d993084aec66fdfbb9` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:10.137468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e742e02-0fbb-45e0-9391-73eeaa10b963 None admin] RBAC: Authorizing `identity:get_user(user_id=3bb65f7f038e40d993084aec66fdfbb9)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:10.139717 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e742e02-0fbb-45e0-9391-73eeaa10b963 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:10.142762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 160/312] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:30:09 2022] GET /identity/v3/users/3bb65f7f038e40d993084aec66fdfbb9 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:10.147540 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e742e02-0fbb-45e0-9391-73eeaa10b963 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:10.232096 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ea4a7d-d129-43d8-81c3-e2e64ba4f330 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:10.274610 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae470d6-3f21-429e-8ffd-dfebc487eae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:10.275264 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:10.276818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae470d6-3f21-429e-8ffd-dfebc487eae4 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:10.277927 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae470d6-3f21-429e-8ffd-dfebc487eae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:10.278663 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae470d6-3f21-429e-8ffd-dfebc487eae4 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:10.279631 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae470d6-3f21-429e-8ffd-dfebc487eae4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:10.280133 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae470d6-3f21-429e-8ffd-dfebc487eae4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:10.281752 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae470d6-3f21-429e-8ffd-dfebc487eae4 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:10.281858 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e742e02-0fbb-45e0-9391-73eeaa10b963 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:10.293038 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:10.298964 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 153/313] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:30:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:10.304774 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae470d6-3f21-429e-8ffd-dfebc487eae4 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:10.333322 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23006ff4-8297-4a73-8229-4e55d5816837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:10.334216 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23006ff4-8297-4a73-8229-4e55d5816837 None admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:10.334507 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23006ff4-8297-4a73-8229-4e55d5816837 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:10.334786 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23006ff4-8297-4a73-8229-4e55d5816837 None admin] PATH_INFO: `/v3/system/users/3bb65f7f038e40d993084aec66fdfbb9/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:10.336352 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23006ff4-8297-4a73-8229-4e55d5816837 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3bb65f7f038e40d993084aec66fdfbb9, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:10.338416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23006ff4-8297-4a73-8229-4e55d5816837 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:10.429425 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae470d6-3f21-429e-8ffd-dfebc487eae4 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:10.440979 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 161/314] 10.222.0.13 () {58 vars in 1334 bytes} [Fri May 27 19:30:10 2022] PUT /identity/v3/system/users/3bb65f7f038e40d993084aec66fdfbb9/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:30:10.471819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa312c-793e-476f-99af-aa5bb3eda3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:10.473425 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa312c-793e-476f-99af-aa5bb3eda3be None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:10.473853 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa312c-793e-476f-99af-aa5bb3eda3be None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:10.474482 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effa312c-793e-476f-99af-aa5bb3eda3be None admin] PATH_INFO: `/v3/users/ea4d03e2ac6c4426951e5cfee25db55d` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:10.476104 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effa312c-793e-476f-99af-aa5bb3eda3be None admin] RBAC: Authorizing `identity:get_user(user_id=ea4d03e2ac6c4426951e5cfee25db55d)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:10.477845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effa312c-793e-476f-99af-aa5bb3eda3be None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:10.480369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 154/315] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:30:10 2022] GET /identity/v3/users/ea4d03e2ac6c4426951e5cfee25db55d => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:10.486334 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23006ff4-8297-4a73-8229-4e55d5816837 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:10.618012 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23006ff4-8297-4a73-8229-4e55d5816837 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:10.659570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a003e-e8bd-4cd7-b7c0-b45201a3aea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:10.660224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a003e-e8bd-4cd7-b7c0-b45201a3aea3 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:10.660506 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a003e-e8bd-4cd7-b7c0-b45201a3aea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:10.676878 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4a003e-e8bd-4cd7-b7c0-b45201a3aea3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:10.694587 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:10.737561 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a003e-e8bd-4cd7-b7c0-b45201a3aea3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:10.739037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4a003e-e8bd-4cd7-b7c0-b45201a3aea3 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:10.852897 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 162/316] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:30:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:30:10.859908 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa312c-793e-476f-99af-aa5bb3eda3be None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:10.932627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:10.974052 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effa312c-793e-476f-99af-aa5bb3eda3be None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:11.010183 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd530d9-ee09-4d99-ae54-13d99b197664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:11.012186 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd530d9-ee09-4d99-ae54-13d99b197664 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:11.012676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd530d9-ee09-4d99-ae54-13d99b197664 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:11.013194 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd530d9-ee09-4d99-ae54-13d99b197664 None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:11.014680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd530d9-ee09-4d99-ae54-13d99b197664 None admin] RBAC: Authorizing `identity:get_project(project_id=7abea439f4624f6680796de4a97fc13b)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:11.016310 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd530d9-ee09-4d99-ae54-13d99b197664 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:11.018573 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 155/317] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:30:10 2022] GET /identity/v3/projects/7abea439f4624f6680796de4a97fc13b => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:11.028984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a003e-e8bd-4cd7-b7c0-b45201a3aea3 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:11.090225 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:11.135105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4a003e-e8bd-4cd7-b7c0-b45201a3aea3 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:11.171279 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f00a87b-b755-4913-bb5e-d10c3d7f3cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:11.171943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00a87b-b755-4913-bb5e-d10c3d7f3cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:11.172228 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00a87b-b755-4913-bb5e-d10c3d7f3cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:11.172504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f00a87b-b755-4913-bb5e-d10c3d7f3cc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:11.173360 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00a87b-b755-4913-bb5e-d10c3d7f3cc2 None admin] RBAC: Adding query filter params (role.id=9170e7d540b446f3b5507688ca335ecc, scope.project.id=7abea439f4624f6680796de4a97fc13b, user.id=ea4d03e2ac6c4426951e5cfee25db55d) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:11.173673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00a87b-b755-4913-bb5e-d10c3d7f3cc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:11.175117 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f00a87b-b755-4913-bb5e-d10c3d7f3cc2 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:11.181707 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 163/318] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:30:11 2022] GET /identity/v3/role_assignments?user.id=ea4d03e2ac6c4426951e5cfee25db55d&scope.project.id=7abea439f4624f6680796de4a97fc13b&role.id=9170e7d540b446f3b5507688ca335ecc => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:11.559909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2520997-7a5b-4657-a2de-d2b14d33d568 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:11.560428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2520997-7a5b-4657-a2de-d2b14d33d568 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:11.560428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2520997-7a5b-4657-a2de-d2b14d33d568 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:11.561998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 156/319] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:11.567084 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f4688-d883-4fb1-8abf-95552e1ac329 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:11.567384 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f4688-d883-4fb1-8abf-95552e1ac329 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:11.567665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f4688-d883-4fb1-8abf-95552e1ac329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:11.585462 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-a92f4688-d883-4fb1-8abf-95552e1ac329 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:11.586645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-a92f4688-d883-4fb1-8abf-95552e1ac329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:11.661943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 164/320] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:11.686133 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2520997-7a5b-4657-a2de-d2b14d33d568 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:11.797704 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2520997-7a5b-4657-a2de-d2b14d33d568 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:11.832538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06f2a1-0d33-49b9-9d26-b0c9ee81c604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:11.833757 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06f2a1-0d33-49b9-9d26-b0c9ee81c604 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:11.834267 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06f2a1-0d33-49b9-9d26-b0c9ee81c604 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:11.834745 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06f2a1-0d33-49b9-9d26-b0c9ee81c604 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:11.836086 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06f2a1-0d33-49b9-9d26-b0c9ee81c604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:11.837773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06f2a1-0d33-49b9-9d26-b0c9ee81c604 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:11.840415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 157/321] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:11.845994 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f4688-d883-4fb1-8abf-95552e1ac329 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:11.856209 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba112b17-286c-4354-b4c2-0b3364fb3f0c None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:11.856566 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba112b17-286c-4354-b4c2-0b3364fb3f0c None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:11.856849 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba112b17-286c-4354-b4c2-0b3364fb3f0c None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:11.858399 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 158/322] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 19:30:11.863186 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe20d1a-69da-4e36-a8f6-e6d70cff47aa None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:11.863479 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe20d1a-69da-4e36-a8f6-e6d70cff47aa None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:11.863762 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe20d1a-69da-4e36-a8f6-e6d70cff47aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:11.871418 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:11.882553 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-abe20d1a-69da-4e36-a8f6-e6d70cff47aa None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:11.883647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-abe20d1a-69da-4e36-a8f6-e6d70cff47aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:11.906665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:11.956006 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 159/323] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 19:30:11.956664 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f4688-d883-4fb1-8abf-95552e1ac329 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:11.978847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe20d1a-69da-4e36-a8f6-e6d70cff47aa None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:11.997693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44606368-b59f-4ac8-83c3-c1e80bf6e7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:11.998367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44606368-b59f-4ac8-83c3-c1e80bf6e7e8 None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:11.998648 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44606368-b59f-4ac8-83c3-c1e80bf6e7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:11.998927 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44606368-b59f-4ac8-83c3-c1e80bf6e7e8 None admin] PATH_INFO: `/v3/groups` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:11.999613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44606368-b59f-4ac8-83c3-c1e80bf6e7e8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:12.001103 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44606368-b59f-4ac8-83c3-c1e80bf6e7e8 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:12.084125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe20d1a-69da-4e36-a8f6-e6d70cff47aa None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:12.113957 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 165/324] 10.222.0.13 () {60 vars in 1134 bytes} [Fri May 27 19:30:11 2022] POST /identity/v3/groups => generated 226 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:30:12.124396 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918c5ad-6ee2-47fb-acd3-603ace032a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:12.125331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918c5ad-6ee2-47fb-acd3-603ace032a58 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:12.125698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918c5ad-6ee2-47fb-acd3-603ace032a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:12.125952 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918c5ad-6ee2-47fb-acd3-603ace032a58 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:12.128264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:12.140122 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5918c5ad-6ee2-47fb-acd3-603ace032a58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:12.150592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-5918c5ad-6ee2-47fb-acd3-603ace032a58 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 19:30:12.150592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:12.150592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:12.150592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:12.150592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:12.150592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:12.150592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:12.150592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:12.150592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:12.150592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:12.150592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:12.150592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:12.150592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:12.150592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:12.150592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:12.150592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:12.152324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:12.152324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:12.152324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:12.152324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:12.152324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:12.152324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:12.152324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:12.152324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:12.152324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:12.152324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:12.152324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:12.152324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:12.152324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:12.152324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:12.152324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:12.152324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:12.153565 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:12.153565 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:12.153565 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:12.153565 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:12.153565 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:12.153565 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:12.153565 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:12.153565 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:12.153565 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:12.153565 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:12.153565 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 19:30:12.153565 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:12.154803 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 160/325] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:12.156456 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44606368-b59f-4ac8-83c3-c1e80bf6e7e8 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:12.269458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44606368-b59f-4ac8-83c3-c1e80bf6e7e8 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:12.284926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:12.307186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eab4c7-3046-46de-a54c-01ab4868530b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:12.308572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eab4c7-3046-46de-a54c-01ab4868530b None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:12.308866 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eab4c7-3046-46de-a54c-01ab4868530b None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:12.309172 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eab4c7-3046-46de-a54c-01ab4868530b None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:12.310267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eab4c7-3046-46de-a54c-01ab4868530b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:12.310267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eab4c7-3046-46de-a54c-01ab4868530b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:12.311638 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eab4c7-3046-46de-a54c-01ab4868530b None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:12.322433 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 166/326] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:30:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:12.328058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918c5ad-6ee2-47fb-acd3-603ace032a58 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:12.444308 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918c5ad-6ee2-47fb-acd3-603ace032a58 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:12.494191 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a2fdc7-7203-49b1-97bd-1a643f0017d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:12.495072 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2fdc7-7203-49b1-97bd-1a643f0017d6 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:12.495414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2fdc7-7203-49b1-97bd-1a643f0017d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:12.496362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a2fdc7-7203-49b1-97bd-1a643f0017d6 None admin] PATH_INFO: `/v3/users/33d779b5c9fd495e8379f3d1f838e6ce` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:12.497645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2fdc7-7203-49b1-97bd-1a643f0017d6 None admin] RBAC: Authorizing `identity:get_user(user_id=33d779b5c9fd495e8379f3d1f838e6ce)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:12.499099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a2fdc7-7203-49b1-97bd-1a643f0017d6 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:12.500991 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 161/327] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:30:12 2022] GET /identity/v3/users/33d779b5c9fd495e8379f3d1f838e6ce => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:12.506776 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eab4c7-3046-46de-a54c-01ab4868530b None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:12.628290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eab4c7-3046-46de-a54c-01ab4868530b None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:12.665624 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546daab5-8855-4b51-bebb-583564e486bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:12.666569 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546daab5-8855-4b51-bebb-583564e486bf None admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:12.666857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546daab5-8855-4b51-bebb-583564e486bf None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:12.667165 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546daab5-8855-4b51-bebb-583564e486bf None admin] PATH_INFO: `/v3/system/users/33d779b5c9fd495e8379f3d1f838e6ce/roles/9170e7d540b446f3b5507688ca335ecc` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:12.669400 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546daab5-8855-4b51-bebb-583564e486bf None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=33d779b5c9fd495e8379f3d1f838e6ce, role_id=9170e7d540b446f3b5507688ca335ecc)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:12.670878 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546daab5-8855-4b51-bebb-583564e486bf None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:12.772911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 167/328] 10.222.0.13 () {58 vars in 1334 bytes} [Fri May 27 19:30:12 2022] PUT /identity/v3/system/users/33d779b5c9fd495e8379f3d1f838e6ce/roles/9170e7d540b446f3b5507688ca335ecc => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:30:12.790980 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b525c47-e38b-4ab0-866e-78ce6a6714ec None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:12.791538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b525c47-e38b-4ab0-866e-78ce6a6714ec None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:12.791538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b525c47-e38b-4ab0-866e-78ce6a6714ec None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:12.793050 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 162/329] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:12.799679 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf515421-cbde-46ac-aaed-dcbd3e0a56ca None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:12.800300 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf515421-cbde-46ac-aaed-dcbd3e0a56ca None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:12.800588 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf515421-cbde-46ac-aaed-dcbd3e0a56ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:12.821644 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-cf515421-cbde-46ac-aaed-dcbd3e0a56ca None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:12.822879 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-cf515421-cbde-46ac-aaed-dcbd3e0a56ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:12.872084 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:12.925638 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 168/330] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:12.952874 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b525c47-e38b-4ab0-866e-78ce6a6714ec None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:13.052001 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:13.107015 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b525c47-e38b-4ab0-866e-78ce6a6714ec None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:13.151286 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef4dae-b20c-4fdd-a17d-a0a9fbdd650c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:13.153113 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef4dae-b20c-4fdd-a17d-a0a9fbdd650c None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:13.153401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef4dae-b20c-4fdd-a17d-a0a9fbdd650c None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:13.153678 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ef4dae-b20c-4fdd-a17d-a0a9fbdd650c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:13.168158 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ef4dae-b20c-4fdd-a17d-a0a9fbdd650c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:13.176147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-29ef4dae-b20c-4fdd-a17d-a0a9fbdd650c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 19:30:13.176147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:13.176147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:13.176147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:13.176147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:13.176147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:13.176147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:13.176147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:13.176147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:13.176147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:13.176147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:13.176147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:13.176147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:13.176147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:13.176147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:13.176147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:13.177227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:13.177227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:13.177227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:13.177227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:13.177227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:13.177227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:13.177227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:13.177227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:13.177227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:13.177227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:13.177227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:13.177227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:13.177227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:13.177227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:13.177227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:13.177227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:13.178045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:13.178045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:13.178045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:13.178045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:13.178045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:13.178045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:13.178045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:13.178045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:13.178045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:13.178045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:13.178045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 19:30:13.178045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:13.178045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 163/331] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:13.185638 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf515421-cbde-46ac-aaed-dcbd3e0a56ca None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:13.365426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf515421-cbde-46ac-aaed-dcbd3e0a56ca None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:13.415702 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe66ec-fd0d-45b2-a023-f8727c299d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:13.416383 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe66ec-fd0d-45b2-a023-f8727c299d1c None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:13.416665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe66ec-fd0d-45b2-a023-f8727c299d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:13.416938 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe66ec-fd0d-45b2-a023-f8727c299d1c None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:13.417831 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe66ec-fd0d-45b2-a023-f8727c299d1c None admin] RBAC: Adding query filter params (name=member) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:13.418205 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe66ec-fd0d-45b2-a023-f8727c299d1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:13.419606 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe66ec-fd0d-45b2-a023-f8727c299d1c None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:13.433795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 169/332] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:30:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:13.439862 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef4dae-b20c-4fdd-a17d-a0a9fbdd650c None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:13.586365 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ef4dae-b20c-4fdd-a17d-a0a9fbdd650c None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:13.618466 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:13.632463 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944f95a3-9a90-4ce4-8704-2cef2439ddab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:13.633376 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f95a3-9a90-4ce4-8704-2cef2439ddab None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:13.635362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f95a3-9a90-4ce4-8704-2cef2439ddab None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:13.635362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944f95a3-9a90-4ce4-8704-2cef2439ddab None admin] PATH_INFO: `/v3/users/3bb65f7f038e40d993084aec66fdfbb9` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:13.635362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f95a3-9a90-4ce4-8704-2cef2439ddab None admin] RBAC: Authorizing `identity:get_user(user_id=3bb65f7f038e40d993084aec66fdfbb9)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:13.636878 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944f95a3-9a90-4ce4-8704-2cef2439ddab None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:13.638725 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 164/333] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:30:13 2022] GET /identity/v3/users/3bb65f7f038e40d993084aec66fdfbb9 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:13.644237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe66ec-fd0d-45b2-a023-f8727c299d1c None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:13.723474 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:13.843713 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe66ec-fd0d-45b2-a023-f8727c299d1c None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:13.893768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe57417e-7523-4372-9f34-ec048052f88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:13.894371 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57417e-7523-4372-9f34-ec048052f88d None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:13.894658 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57417e-7523-4372-9f34-ec048052f88d None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:13.894941 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe57417e-7523-4372-9f34-ec048052f88d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:13.895803 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57417e-7523-4372-9f34-ec048052f88d None admin] RBAC: Adding query filter params (role.id=cb43a575e47147c799eec2f362f4fe0e, user.id=3bb65f7f038e40d993084aec66fdfbb9, scope.system=all) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:13.896120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57417e-7523-4372-9f34-ec048052f88d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:13.897780 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe57417e-7523-4372-9f34-ec048052f88d None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:13.911585 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 170/334] 10.222.0.13 () {56 vars in 1308 bytes} [Fri May 27 19:30:13 2022] GET /identity/v3/role_assignments?user.id=3bb65f7f038e40d993084aec66fdfbb9&scope.system=all&role.id=cb43a575e47147c799eec2f362f4fe0e => generated 510 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:15.232109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228564a-5ccf-4322-ac9e-638ccf92a174 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:15.232628 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228564a-5ccf-4322-ac9e-638ccf92a174 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:15.232704 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6228564a-5ccf-4322-ac9e-638ccf92a174 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:15.234239 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 165/335] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 19:30:15.240495 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90f2c4-7879-47ac-a30c-95868f76b1c7 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:15.241470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90f2c4-7879-47ac-a30c-95868f76b1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:15.241963 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90f2c4-7879-47ac-a30c-95868f76b1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:15.266060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-6c90f2c4-7879-47ac-a30c-95868f76b1c7 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:15.267274 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-6c90f2c4-7879-47ac-a30c-95868f76b1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:15.360071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 171/336] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:15.394954 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6228564a-5ccf-4322-ac9e-638ccf92a174 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:15.574191 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6228564a-5ccf-4322-ac9e-638ccf92a174 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:15.638012 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd3d3c-20f9-45d8-8225-206a14704704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:15.638886 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd3d3c-20f9-45d8-8225-206a14704704 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:15.639193 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd3d3c-20f9-45d8-8225-206a14704704 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:15.639473 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd3d3c-20f9-45d8-8225-206a14704704 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:15.657180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd3d3c-20f9-45d8-8225-206a14704704 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:15.665561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-c7bd3d3c-20f9-45d8-8225-206a14704704 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 19:30:15.665561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:15.665561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:15.665561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:15.665561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:15.665561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:15.665561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:15.665561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:15.665561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:15.665561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:15.665561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:15.665561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:15.665561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:15.665561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:15.665561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:15.665561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:15.666497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:15.666497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:15.666497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:15.666497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:15.666497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:15.666497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:15.666497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:15.666497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:15.666497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:15.666497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:15.666497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:15.666497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:15.666497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:15.666497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:15.666497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:15.666497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:15.668415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:15.668415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:15.668415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:15.668415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:15.668415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:15.668415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:15.668415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:15.668415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:15.668415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:15.668415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:15.668415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 19:30:15.668415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:15.668415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 166/337] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:15.673891 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90f2c4-7879-47ac-a30c-95868f76b1c7 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:15.733446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:15.888288 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90f2c4-7879-47ac-a30c-95868f76b1c7 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:15.937450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a5fcd-8e3c-4617-b9cb-88c928cde3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:15.938169 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a5fcd-8e3c-4617-b9cb-88c928cde3c2 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:15.938478 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a5fcd-8e3c-4617-b9cb-88c928cde3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:15.938762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a5fcd-8e3c-4617-b9cb-88c928cde3c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:15.939550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a5fcd-8e3c-4617-b9cb-88c928cde3c2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:15.939860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a5fcd-8e3c-4617-b9cb-88c928cde3c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:15.941541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a5fcd-8e3c-4617-b9cb-88c928cde3c2 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:15.957701 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 172/338] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:30:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:15.967027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd3d3c-20f9-45d8-8225-206a14704704 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:16.061374 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:16.094496 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428a415-7441-43d1-83e9-75e581e93aa8 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:16.094847 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428a415-7441-43d1-83e9-75e581e93aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:16.095084 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d428a415-7441-43d1-83e9-75e581e93aa8 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:16.096905 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 173/339] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:16.102713 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f03c6-b6ac-45b2-99aa-d36888e9b732 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:16.103040 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f03c6-b6ac-45b2-99aa-d36888e9b732 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:16.103349 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3f03c6-b6ac-45b2-99aa-d36888e9b732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:16.137671 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd3d3c-20f9-45d8-8225-206a14704704 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:16.144920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-6e3f03c6-b6ac-45b2-99aa-d36888e9b732 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:16.146083 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-6e3f03c6-b6ac-45b2-99aa-d36888e9b732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:16.222454 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c96e16-f401-4a6e-b227-aac4e851b69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:16.223129 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c96e16-f401-4a6e-b227-aac4e851b69a None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:16.224541 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c96e16-f401-4a6e-b227-aac4e851b69a None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:16.224834 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c96e16-f401-4a6e-b227-aac4e851b69a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:16.346108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 174/340] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 19:30:16.363964 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c96e16-f401-4a6e-b227-aac4e851b69a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:16.367191 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c96e16-f401-4a6e-b227-aac4e851b69a None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:16.386806 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f03c6-b6ac-45b2-99aa-d36888e9b732 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:16.412581 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:16.490143 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:16.557212 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3f03c6-b6ac-45b2-99aa-d36888e9b732 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:16.608069 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e919f1-47b1-446e-9a01-300b8aa13927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:16.609254 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e919f1-47b1-446e-9a01-300b8aa13927 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:16.609708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e919f1-47b1-446e-9a01-300b8aa13927 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:16.610161 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e919f1-47b1-446e-9a01-300b8aa13927 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:16.634500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e919f1-47b1-446e-9a01-300b8aa13927 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:16.644984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-63e919f1-47b1-446e-9a01-300b8aa13927 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:30:16.644984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:16.644984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:16.644984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:16.644984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:16.644984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:16.644984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:16.644984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:16.644984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:16.644984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:16.644984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:16.644984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:16.644984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:16.644984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:16.644984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:16.644984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:16.646200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:16.646200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:16.646200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:16.646200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:16.646200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:16.646200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:16.646200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:16.646200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:16.646200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:16.646200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:16.646200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:16.646200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:16.646200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:16.646200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:16.646200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:16.646200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:16.647108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:16.647108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:16.647108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:16.647108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:16.647108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:16.647108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:16.647108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:16.647108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:16.647108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:16.647108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:16.647108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:30:16.647108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:16.648644 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 167/341] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:30:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 19:30:16.649651 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 175/342] 10.222.0.13 () {56 vars in 1114 bytes} [Fri May 27 19:30:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:30:16.655461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c96e16-f401-4a6e-b227-aac4e851b69a None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:16.663003 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e919f1-47b1-446e-9a01-300b8aa13927 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:16.778414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:16.821425 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e919f1-47b1-446e-9a01-300b8aa13927 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:16.874947 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99dad9c-1c34-42a4-aaca-06ad40c16962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:16.875876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99dad9c-1c34-42a4-aaca-06ad40c16962 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:16.876302 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99dad9c-1c34-42a4-aaca-06ad40c16962 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:16.876742 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99dad9c-1c34-42a4-aaca-06ad40c16962 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:16.877868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99dad9c-1c34-42a4-aaca-06ad40c16962 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:16.879651 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99dad9c-1c34-42a4-aaca-06ad40c16962 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:16.881276 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99dad9c-1c34-42a4-aaca-06ad40c16962 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:16.901019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 176/343] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:30:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:16.908355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99dad9c-1c34-42a4-aaca-06ad40c16962 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:16.920246 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c96e16-f401-4a6e-b227-aac4e851b69a None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:17.001803 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350ca8db-6a68-4444-a136-b49c2e210a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:17.003570 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ca8db-6a68-4444-a136-b49c2e210a8b None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:17.004001 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ca8db-6a68-4444-a136-b49c2e210a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:17.004433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350ca8db-6a68-4444-a136-b49c2e210a8b None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:17.006323 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ca8db-6a68-4444-a136-b49c2e210a8b None admin] RBAC: Authorizing `identity:get_project(project_id=b30f38a32f6d4982bd829828ccac6fa7)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:17.007952 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350ca8db-6a68-4444-a136-b49c2e210a8b None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:17.011588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 168/344] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:30:16 2022] GET /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7 => generated 298 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:17.013542 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d2146-dd8f-4fbf-8a5e-2e4261890d5e None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:17.014004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d2146-dd8f-4fbf-8a5e-2e4261890d5e None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:17.014435 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d2146-dd8f-4fbf-8a5e-2e4261890d5e None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:17.016222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 169/345] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:17.022375 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d2146-dd8f-4fbf-8a5e-2e4261890d5e None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:17.065407 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:17.164387 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99dad9c-1c34-42a4-aaca-06ad40c16962 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:17.212368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d2146-dd8f-4fbf-8a5e-2e4261890d5e None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:17.263869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f7a450-02a0-4c73-8f9a-720f4147af50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:17.265118 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7a450-02a0-4c73-8f9a-720f4147af50 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:17.265466 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a967ec1e-4bd9-48e6-8590-785091c116c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:17.265751 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7a450-02a0-4c73-8f9a-720f4147af50 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:17.266168 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f7a450-02a0-4c73-8f9a-720f4147af50 None admin] PATH_INFO: `/v3/groups/0dc59d8cece14689b7d92b31c02b0986` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:17.266829 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967ec1e-4bd9-48e6-8590-785091c116c9 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:17.267379 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967ec1e-4bd9-48e6-8590-785091c116c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:17.267920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a967ec1e-4bd9-48e6-8590-785091c116c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:17.274415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7a450-02a0-4c73-8f9a-720f4147af50 None admin] RBAC: Authorizing `identity:get_group(group_id=0dc59d8cece14689b7d92b31c02b0986)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:17.274928 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:17.277828 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f7a450-02a0-4c73-8f9a-720f4147af50 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:17.280151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 170/346] 10.222.0.13 () {56 vars in 1180 bytes} [Fri May 27 19:30:17 2022] GET /identity/v3/groups/0dc59d8cece14689b7d92b31c02b0986 => generated 226 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:17.281644 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9914d-f659-4197-bf79-1dfc88ec1236 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:17.282076 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9914d-f659-4197-bf79-1dfc88ec1236 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:17.293926 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9914d-f659-4197-bf79-1dfc88ec1236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:17.304705 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-8df9914d-f659-4197-bf79-1dfc88ec1236 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:17.306325 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-8df9914d-f659-4197-bf79-1dfc88ec1236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:17.425936 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 171/347] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:17.427166 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9914d-f659-4197-bf79-1dfc88ec1236 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:17.430415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a967ec1e-4bd9-48e6-8590-785091c116c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:17.433308 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a967ec1e-4bd9-48e6-8590-785091c116c9 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:17.475116 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:17.582721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 177/348] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:30:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:30:17.588980 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622190f6-104c-4cfa-a55e-88cb4aa1f435 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:17.588980 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622190f6-104c-4cfa-a55e-88cb4aa1f435 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:17.588980 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622190f6-104c-4cfa-a55e-88cb4aa1f435 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:17.588980 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 178/349] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:17.594078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622190f6-104c-4cfa-a55e-88cb4aa1f435 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:17.622422 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9914d-f659-4197-bf79-1dfc88ec1236 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:17.657039 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:17.685112 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6141acd-c2df-4582-bcca-12defef6b314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:17.686332 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6141acd-c2df-4582-bcca-12defef6b314 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:17.686864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6141acd-c2df-4582-bcca-12defef6b314 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:17.687146 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6141acd-c2df-4582-bcca-12defef6b314 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:17.688059 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6141acd-c2df-4582-bcca-12defef6b314 None admin] RBAC: Adding query filter params (group.id=0dc59d8cece14689b7d92b31c02b0986, role.id=cb43a575e47147c799eec2f362f4fe0e, scope.project.id=b30f38a32f6d4982bd829828ccac6fa7) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:17.688369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6141acd-c2df-4582-bcca-12defef6b314 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:17.690959 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6141acd-c2df-4582-bcca-12defef6b314 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:17.699245 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 172/350] 10.222.0.13 () {56 vars in 1376 bytes} [Fri May 27 19:30:17 2022] GET /identity/v3/role_assignments?group.id=0dc59d8cece14689b7d92b31c02b0986&scope.project.id=b30f38a32f6d4982bd829828ccac6fa7&role.id=cb43a575e47147c799eec2f362f4fe0e => generated 260 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:17.700658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e53e46-a905-4d68-82e1-8fdbc4c4e37e None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:17.701378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e53e46-a905-4d68-82e1-8fdbc4c4e37e None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:17.701378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e53e46-a905-4d68-82e1-8fdbc4c4e37e None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:17.702683 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 173/351] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:17.703235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e53e46-a905-4d68-82e1-8fdbc4c4e37e None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:17.779259 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622190f6-104c-4cfa-a55e-88cb4aa1f435 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:17.840480 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:17.845108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a8874-6be7-4c36-aae9-b64c825b0c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:17.846096 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a8874-6be7-4c36-aae9-b64c825b0c2b None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:17.846614 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a8874-6be7-4c36-aae9-b64c825b0c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:17.847325 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a8874-6be7-4c36-aae9-b64c825b0c2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:17.862992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a8874-6be7-4c36-aae9-b64c825b0c2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:17.872021 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e53e46-a905-4d68-82e1-8fdbc4c4e37e None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:17.873310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-8d1a8874-6be7-4c36-aae9-b64c825b0c2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 19:30:17.873310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:17.873310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:17.873310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:17.873310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:17.873310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:17.873310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:17.873310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:17.873310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:17.873310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:17.873310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:17.873310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:17.873310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:17.873310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:17.873310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:17.873310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:17.874356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:17.874356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:17.874356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:17.874356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:17.874356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:17.874356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:17.874356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:17.874356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:17.874356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:17.874356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:17.874356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:17.874356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:17.874356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:17.874356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:17.874356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:17.874356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:17.875177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:17.875177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:17.875177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:17.875177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:17.875177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:17.875177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:17.875177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:17.875177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:17.875177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:17.875177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:17.875177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 19:30:17.875177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:17.876252 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 179/352] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:17.877645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1910e0-fd6e-4d58-a307-886b68eea653 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:17.877946 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1910e0-fd6e-4d58-a307-886b68eea653 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:17.878283 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1910e0-fd6e-4d58-a307-886b68eea653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:17.902493 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-4c1910e0-fd6e-4d58-a307-886b68eea653 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:17.904007 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-4c1910e0-fd6e-4d58-a307-886b68eea653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:17.927963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20215f-ff9e-45f1-be1d-a59a8401702b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:17.929103 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20215f-ff9e-45f1-be1d-a59a8401702b None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:17.929392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20215f-ff9e-45f1-be1d-a59a8401702b None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:17.929751 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20215f-ff9e-45f1-be1d-a59a8401702b None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:17.931109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20215f-ff9e-45f1-be1d-a59a8401702b None admin] RBAC: Authorizing `identity:get_project(project_id=b30f38a32f6d4982bd829828ccac6fa7)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:17.933538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20215f-ff9e-45f1-be1d-a59a8401702b None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:17.935709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 174/353] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:30:17 2022] GET /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:17.937212 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af22cbf-b8c6-4eb2-b59c-977f29d78636 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:17.937475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af22cbf-b8c6-4eb2-b59c-977f29d78636 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:17.937788 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af22cbf-b8c6-4eb2-b59c-977f29d78636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:17.958613 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-2af22cbf-b8c6-4eb2-b59c-977f29d78636 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:17.959998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-2af22cbf-b8c6-4eb2-b59c-977f29d78636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:18.016246 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 180/354] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:18.016983 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1910e0-fd6e-4d58-a307-886b68eea653 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:18.065761 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:18.085331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 175/355] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:18.089703 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af22cbf-b8c6-4eb2-b59c-977f29d78636 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:18.215632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:18.223348 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1910e0-fd6e-4d58-a307-886b68eea653 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:18.255376 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af22cbf-b8c6-4eb2-b59c-977f29d78636 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:18.296246 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b976c-d3dc-4576-96e8-435e7ac037e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:18.300994 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b976c-d3dc-4576-96e8-435e7ac037e8 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:18.300994 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b976c-d3dc-4576-96e8-435e7ac037e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:18.300994 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b976c-d3dc-4576-96e8-435e7ac037e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:18.300994 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b976c-d3dc-4576-96e8-435e7ac037e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:18.300994 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b976c-d3dc-4576-96e8-435e7ac037e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:18.304008 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4b976c-d3dc-4576-96e8-435e7ac037e8 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:18.316595 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e629fad-36ed-49ce-8571-5897fc3d7170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:18.317791 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e629fad-36ed-49ce-8571-5897fc3d7170 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:18.318226 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e629fad-36ed-49ce-8571-5897fc3d7170 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:18.318373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 181/356] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:30:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:18.318682 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e629fad-36ed-49ce-8571-5897fc3d7170 None admin] PATH_INFO: `/v3/groups/0dc59d8cece14689b7d92b31c02b0986` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:18.319584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b976c-d3dc-4576-96e8-435e7ac037e8 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:18.320404 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e629fad-36ed-49ce-8571-5897fc3d7170 None admin] RBAC: Authorizing `identity:get_group(group_id=0dc59d8cece14689b7d92b31c02b0986)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:18.323366 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e629fad-36ed-49ce-8571-5897fc3d7170 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:18.326452 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 176/357] 10.222.0.13 () {56 vars in 1180 bytes} [Fri May 27 19:30:18 2022] GET /identity/v3/groups/0dc59d8cece14689b7d92b31c02b0986 => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:18.327798 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e629fad-36ed-49ce-8571-5897fc3d7170 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:18.488853 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:18.507509 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b976c-d3dc-4576-96e8-435e7ac037e8 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:18.521269 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e629fad-36ed-49ce-8571-5897fc3d7170 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:18.572455 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bb6c7-a2c7-484d-b480-442cdf0fa5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:18.573261 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bb6c7-a2c7-484d-b480-442cdf0fa5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:18.574388 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bb6c7-a2c7-484d-b480-442cdf0fa5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:18.574773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bb6c7-a2c7-484d-b480-442cdf0fa5e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:18.582980 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867043c-e902-4c0f-a290-5b262ce6736e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:18.584071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867043c-e902-4c0f-a290-5b262ce6736e None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:18.584699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867043c-e902-4c0f-a290-5b262ce6736e None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:18.585048 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867043c-e902-4c0f-a290-5b262ce6736e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:18.594492 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6bb6c7-a2c7-484d-b480-442cdf0fa5e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:18.602895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-1e6bb6c7-a2c7-484d-b480-442cdf0fa5e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:30:18.602895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:18.602895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:18.602895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:18.602895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:18.602895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:18.602895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:18.602895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:18.602895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:18.602895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:18.602895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:18.602895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:18.602895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:18.602895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:18.602895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:18.602895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:18.604004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:18.604004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:18.604004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:18.604004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:18.604004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:18.604004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:18.604004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:18.604004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:18.604004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:18.604004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:18.604004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:18.604004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:18.604004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:18.604004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:18.604004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:18.604004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:18.605584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:18.605584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:18.605584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:18.605584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:18.605584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:18.605584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:18.605584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:18.605584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:18.605584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:18.605584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:18.605584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:30:18.605584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:18.605584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 177/358] 10.222.0.13 () {56 vars in 1114 bytes} [Fri May 27 19:30:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:18.606793 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46776afd-4ead-45f2-a720-bdedae797649 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:18.606793 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46776afd-4ead-45f2-a720-bdedae797649 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:18.606793 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46776afd-4ead-45f2-a720-bdedae797649 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:18.609144 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 178/359] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:18.609144 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46776afd-4ead-45f2-a720-bdedae797649 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:18.610908 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867043c-e902-4c0f-a290-5b262ce6736e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:18.611458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:18.628267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-c867043c-e902-4c0f-a290-5b262ce6736e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 19:30:18.628267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:18.628267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:18.628267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:18.628267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:18.628267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:18.628267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:18.628267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:18.628267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:18.628267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:18.628267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:18.628267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:18.628267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:18.628267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:18.628267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:18.628267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:18.630426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:18.630426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:18.630426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:18.630426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:18.630426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:18.630426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:18.630426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:18.630426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:18.630426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:18.630426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:18.630426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:18.630426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:18.630426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:18.630426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:18.630426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:18.630426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:18.632450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:18.632450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:18.632450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:18.632450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:18.632450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:18.632450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:18.632450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:18.632450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:18.632450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:18.632450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:18.632450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 19:30:18.632450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:18.632450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 182/360] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:18.632450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867043c-e902-4c0f-a290-5b262ce6736e None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:18.793098 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46776afd-4ead-45f2-a720-bdedae797649 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:18.809320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867043c-e902-4c0f-a290-5b262ce6736e None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:18.836063 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df77314-4a47-43e7-9880-3356416fe94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:18.836652 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df77314-4a47-43e7-9880-3356416fe94b None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:18.837076 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df77314-4a47-43e7-9880-3356416fe94b None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:18.837418 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df77314-4a47-43e7-9880-3356416fe94b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:18.867078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48897d-0c01-4e06-9cf7-94a89643170d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:18.867078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48897d-0c01-4e06-9cf7-94a89643170d None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:18.867078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48897d-0c01-4e06-9cf7-94a89643170d None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:18.867078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48897d-0c01-4e06-9cf7-94a89643170d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:18.867485 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48897d-0c01-4e06-9cf7-94a89643170d None admin] RBAC: Adding query filter params (group.id=0dc59d8cece14689b7d92b31c02b0986, role.id=eda4ac05bc364224a7f319e7ded54073, scope.project.id=b30f38a32f6d4982bd829828ccac6fa7) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:18.867485 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48897d-0c01-4e06-9cf7-94a89643170d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:18.868970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48897d-0c01-4e06-9cf7-94a89643170d None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:18.877049 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 183/361] 10.222.0.13 () {56 vars in 1376 bytes} [Fri May 27 19:30:18 2022] GET /identity/v3/role_assignments?group.id=0dc59d8cece14689b7d92b31c02b0986&scope.project.id=b30f38a32f6d4982bd829828ccac6fa7&role.id=eda4ac05bc364224a7f319e7ded54073 => generated 260 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:18.879063 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf063540-b6a2-42bf-8f57-cf8903f151a9 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:18.880479 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf063540-b6a2-42bf-8f57-cf8903f151a9 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:18.881077 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf063540-b6a2-42bf-8f57-cf8903f151a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:18.910991 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-bf063540-b6a2-42bf-8f57-cf8903f151a9 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:18.913430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-bf063540-b6a2-42bf-8f57-cf8903f151a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:18.918466 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:18.956759 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df77314-4a47-43e7-9880-3356416fe94b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:18.958748 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df77314-4a47-43e7-9880-3356416fe94b None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:18.974339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:19.017110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 184/362] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:19.021253 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf063540-b6a2-42bf-8f57-cf8903f151a9 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:19.146354 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 179/363] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:30:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:30:19.148270 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df77314-4a47-43e7-9880-3356416fe94b None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:19.167864 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf063540-b6a2-42bf-8f57-cf8903f151a9 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:19.218819 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964c468-c2ec-4667-bba6-f5927d8011b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:19.219863 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964c468-c2ec-4667-bba6-f5927d8011b2 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:19.220575 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964c468-c2ec-4667-bba6-f5927d8011b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:19.221426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964c468-c2ec-4667-bba6-f5927d8011b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:19.222569 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964c468-c2ec-4667-bba6-f5927d8011b2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:19.223083 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964c468-c2ec-4667-bba6-f5927d8011b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:19.224748 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964c468-c2ec-4667-bba6-f5927d8011b2 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:19.239375 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 185/364] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:30:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:19.240691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964c468-c2ec-4667-bba6-f5927d8011b2 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:19.248989 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:19.319252 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df77314-4a47-43e7-9880-3356416fe94b None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:19.327634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:19.357945 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78231919-9837-464c-a293-dd8d85761f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:19.358634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78231919-9837-464c-a293-dd8d85761f21 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:19.358914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78231919-9837-464c-a293-dd8d85761f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:19.359187 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78231919-9837-464c-a293-dd8d85761f21 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:19.359946 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78231919-9837-464c-a293-dd8d85761f21 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:19.360281 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78231919-9837-464c-a293-dd8d85761f21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:19.362500 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78231919-9837-464c-a293-dd8d85761f21 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:19.372522 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 180/365] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:30:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:19.373483 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78231919-9837-464c-a293-dd8d85761f21 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:19.416405 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964c468-c2ec-4667-bba6-f5927d8011b2 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:19.474636 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba7b23-0110-4967-8c6a-1686fd96de28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:19.475535 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba7b23-0110-4967-8c6a-1686fd96de28 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:19.475886 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba7b23-0110-4967-8c6a-1686fd96de28 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:19.476174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ba7b23-0110-4967-8c6a-1686fd96de28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:19.491737 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ba7b23-0110-4967-8c6a-1686fd96de28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:19.502442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-38ba7b23-0110-4967-8c6a-1686fd96de28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:30:19.502442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:19.502442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:19.502442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:19.502442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:19.502442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:19.502442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:19.502442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:19.502442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:19.502442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:19.502442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:19.502442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:19.502442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:19.502442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:19.502442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:19.502442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:19.503426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:19.503426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:19.503426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:19.503426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:19.503426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:19.503426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:19.503426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:19.503426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:19.503426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:19.503426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:19.503426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:19.503426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:19.503426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:19.503426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:19.503426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:19.503426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:19.504239 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:19.504239 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:19.504239 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:19.504239 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:19.504239 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:19.504239 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:19.504239 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:19.504239 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:19.504239 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:19.504239 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:19.504239 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:30:19.504239 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:19.505394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 186/366] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:30:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:19.505698 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba7b23-0110-4967-8c6a-1686fd96de28 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:19.548139 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78231919-9837-464c-a293-dd8d85761f21 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:19.606116 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:19.606763 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc861be-eacc-4828-aa8d-c5e05b983f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:19.607721 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc861be-eacc-4828-aa8d-c5e05b983f85 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:19.608006 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc861be-eacc-4828-aa8d-c5e05b983f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:19.608280 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc861be-eacc-4828-aa8d-c5e05b983f85 None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:19.609959 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc861be-eacc-4828-aa8d-c5e05b983f85 None admin] RBAC: Authorizing `identity:get_project(project_id=7abea439f4624f6680796de4a97fc13b)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:19.612326 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc861be-eacc-4828-aa8d-c5e05b983f85 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:19.615518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 181/367] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:30:19 2022] GET /identity/v3/projects/7abea439f4624f6680796de4a97fc13b => generated 302 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:19.616756 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc861be-eacc-4828-aa8d-c5e05b983f85 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:19.648484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ba7b23-0110-4967-8c6a-1686fd96de28 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:19.657909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:19.704315 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a46666-dc39-4380-94ec-c09e805fd34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:19.705815 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a46666-dc39-4380-94ec-c09e805fd34f None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:19.706309 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a46666-dc39-4380-94ec-c09e805fd34f None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:19.706806 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a46666-dc39-4380-94ec-c09e805fd34f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:19.819591 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc861be-eacc-4828-aa8d-c5e05b983f85 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:19.821550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a46666-dc39-4380-94ec-c09e805fd34f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:19.825858 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a46666-dc39-4380-94ec-c09e805fd34f None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:19.896869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51da892a-b13e-4198-8291-819eabfed63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:19.898349 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da892a-b13e-4198-8291-819eabfed63e None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:19.898839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da892a-b13e-4198-8291-819eabfed63e None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:19.899472 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51da892a-b13e-4198-8291-819eabfed63e None admin] PATH_INFO: `/v3/users/33d779b5c9fd495e8379f3d1f838e6ce` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:19.901294 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51da892a-b13e-4198-8291-819eabfed63e None admin] RBAC: Authorizing `identity:get_user(user_id=33d779b5c9fd495e8379f3d1f838e6ce)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:19.903775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51da892a-b13e-4198-8291-819eabfed63e None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:19.905809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 182/368] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:30:19 2022] GET /identity/v3/users/33d779b5c9fd495e8379f3d1f838e6ce => generated 293 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:19.907081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51da892a-b13e-4198-8291-819eabfed63e None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:19.931659 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:19.992649 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 187/369] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:30:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:30:19.993495 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a46666-dc39-4380-94ec-c09e805fd34f None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:20.106824 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:20.111624 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51da892a-b13e-4198-8291-819eabfed63e None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:20.154663 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a46666-dc39-4380-94ec-c09e805fd34f None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:20.159179 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635bf397-374c-43bc-a3c4-4a0ab1113c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:20.159883 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635bf397-374c-43bc-a3c4-4a0ab1113c94 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:20.160219 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635bf397-374c-43bc-a3c4-4a0ab1113c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:20.160526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635bf397-374c-43bc-a3c4-4a0ab1113c94 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:20.161429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635bf397-374c-43bc-a3c4-4a0ab1113c94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:20.161766 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635bf397-374c-43bc-a3c4-4a0ab1113c94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:20.163456 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635bf397-374c-43bc-a3c4-4a0ab1113c94 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:20.173525 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 183/370] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:20.174553 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635bf397-374c-43bc-a3c4-4a0ab1113c94 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:20.204035 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa45dab-c506-42ad-9cec-8f191a2d96fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:20.208637 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa45dab-c506-42ad-9cec-8f191a2d96fb None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:20.208637 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa45dab-c506-42ad-9cec-8f191a2d96fb None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:20.208637 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa45dab-c506-42ad-9cec-8f191a2d96fb None admin] PATH_INFO: `/v3/groups/0dc59d8cece14689b7d92b31c02b0986` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:20.214609 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa45dab-c506-42ad-9cec-8f191a2d96fb None admin] RBAC: Authorizing `identity:get_group(group_id=0dc59d8cece14689b7d92b31c02b0986)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:20.216127 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa45dab-c506-42ad-9cec-8f191a2d96fb None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:20.217936 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 188/371] 10.222.0.13 () {56 vars in 1180 bytes} [Fri May 27 19:30:19 2022] GET /identity/v3/groups/0dc59d8cece14689b7d92b31c02b0986 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:20.218742 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa45dab-c506-42ad-9cec-8f191a2d96fb None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:20.266187 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:20.350663 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635bf397-374c-43bc-a3c4-4a0ab1113c94 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:20.386485 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa45dab-c506-42ad-9cec-8f191a2d96fb None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:20.413127 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45003a78-b857-4f38-8a19-cf46ff3513b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:20.414096 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45003a78-b857-4f38-8a19-cf46ff3513b3 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:20.414541 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45003a78-b857-4f38-8a19-cf46ff3513b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:20.414969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45003a78-b857-4f38-8a19-cf46ff3513b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:20.415967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45003a78-b857-4f38-8a19-cf46ff3513b3 None admin] RBAC: Adding query filter params (role.id=9170e7d540b446f3b5507688ca335ecc, user.id=33d779b5c9fd495e8379f3d1f838e6ce, scope.system=all) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:20.416551 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45003a78-b857-4f38-8a19-cf46ff3513b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:20.421147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45003a78-b857-4f38-8a19-cf46ff3513b3 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:20.436632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 184/372] 10.222.0.13 () {56 vars in 1308 bytes} [Fri May 27 19:30:20 2022] GET /identity/v3/role_assignments?user.id=33d779b5c9fd495e8379f3d1f838e6ce&scope.system=all&role.id=9170e7d540b446f3b5507688ca335ecc => generated 510 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:20.437927 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45003a78-b857-4f38-8a19-cf46ff3513b3 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:20.446847 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8355a-339b-42c1-8399-e1da77abdd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:20.448146 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8355a-339b-42c1-8399-e1da77abdd53 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:20.448652 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8355a-339b-42c1-8399-e1da77abdd53 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:20.449079 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8355a-339b-42c1-8399-e1da77abdd53 None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:20.450502 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8355a-339b-42c1-8399-e1da77abdd53 None admin] RBAC: Authorizing `identity:get_project(project_id=7abea439f4624f6680796de4a97fc13b)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:20.452083 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8355a-339b-42c1-8399-e1da77abdd53 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:20.454320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 189/373] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:30:20 2022] GET /identity/v3/projects/7abea439f4624f6680796de4a97fc13b => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:20.454901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8355a-339b-42c1-8399-e1da77abdd53 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:20.482678 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:20.599901 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45003a78-b857-4f38-8a19-cf46ff3513b3 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:20.648462 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdbfb61-915a-4a1b-b59a-6eba8f601b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:20.649620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbfb61-915a-4a1b-b59a-6eba8f601b77 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:20.650135 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbfb61-915a-4a1b-b59a-6eba8f601b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:20.651018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdbfb61-915a-4a1b-b59a-6eba8f601b77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:20.657469 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:20.663287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8355a-339b-42c1-8399-e1da77abdd53 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:20.711256 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8423bac-1db3-4ea5-9b3b-c4db47516363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:20.712575 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8423bac-1db3-4ea5-9b3b-c4db47516363 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:20.714389 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8423bac-1db3-4ea5-9b3b-c4db47516363 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:20.714945 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8423bac-1db3-4ea5-9b3b-c4db47516363 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:20.716160 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8423bac-1db3-4ea5-9b3b-c4db47516363 None admin] RBAC: Adding query filter params (group.id=0dc59d8cece14689b7d92b31c02b0986, role.id=cb43a575e47147c799eec2f362f4fe0e, scope.project.id=7abea439f4624f6680796de4a97fc13b) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:20.716825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8423bac-1db3-4ea5-9b3b-c4db47516363 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:20.719597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8423bac-1db3-4ea5-9b3b-c4db47516363 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:20.728173 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 190/374] 10.222.0.13 () {56 vars in 1376 bytes} [Fri May 27 19:30:20 2022] GET /identity/v3/role_assignments?group.id=0dc59d8cece14689b7d92b31c02b0986&scope.project.id=7abea439f4624f6680796de4a97fc13b&role.id=cb43a575e47147c799eec2f362f4fe0e => generated 260 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:20.733786 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8423bac-1db3-4ea5-9b3b-c4db47516363 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:20.756549 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdbfb61-915a-4a1b-b59a-6eba8f601b77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:20.758282 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdbfb61-915a-4a1b-b59a-6eba8f601b77 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:20.788695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:20.902196 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8423bac-1db3-4ea5-9b3b-c4db47516363 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:20.929703 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 185/375] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:30:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:30:20.932204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a7b55-935e-44a7-afbf-1ea36a6dfbe8 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:20.932734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a7b55-935e-44a7-afbf-1ea36a6dfbe8 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:20.933306 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a7b55-935e-44a7-afbf-1ea36a6dfbe8 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:20.936845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 186/376] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:20.939444 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a7b55-935e-44a7-afbf-1ea36a6dfbe8 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:20.948296 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc38d9-39a1-4f3f-8cb8-bf890954a614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:20.949680 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc38d9-39a1-4f3f-8cb8-bf890954a614 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:20.950194 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc38d9-39a1-4f3f-8cb8-bf890954a614 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:20.950726 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc38d9-39a1-4f3f-8cb8-bf890954a614 None admin] PATH_INFO: `/v3/groups/0dc59d8cece14689b7d92b31c02b0986` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:20.952425 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc38d9-39a1-4f3f-8cb8-bf890954a614 None admin] RBAC: Authorizing `identity:get_group(group_id=0dc59d8cece14689b7d92b31c02b0986)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:20.954289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc38d9-39a1-4f3f-8cb8-bf890954a614 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:20.956763 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 191/377] 10.222.0.13 () {56 vars in 1180 bytes} [Fri May 27 19:30:20 2022] GET /identity/v3/groups/0dc59d8cece14689b7d92b31c02b0986 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:20.958556 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2745246-87ae-4701-a276-4cba069c15dd None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:20.959059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2745246-87ae-4701-a276-4cba069c15dd None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:20.959549 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2745246-87ae-4701-a276-4cba069c15dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:20.985397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-e2745246-87ae-4701-a276-4cba069c15dd None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:20.987518 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-e2745246-87ae-4701-a276-4cba069c15dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:21.028038 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:21.095636 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a7b55-935e-44a7-afbf-1ea36a6dfbe8 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:21.103749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 192/378] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:21.105699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2745246-87ae-4701-a276-4cba069c15dd None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:21.125216 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:21.145242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c306a-59b1-4d48-8f34-aae4618abe21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:21.146233 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c306a-59b1-4d48-8f34-aae4618abe21 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:21.146521 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c306a-59b1-4d48-8f34-aae4618abe21 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:21.146801 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1c306a-59b1-4d48-8f34-aae4618abe21 None admin] PATH_INFO: `/v3/projects/6c2fc958121b4494a2ef337a1cb0b9b4` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:21.148128 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c306a-59b1-4d48-8f34-aae4618abe21 None admin] RBAC: Authorizing `identity:get_project(project_id=6c2fc958121b4494a2ef337a1cb0b9b4)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:21.152504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1c306a-59b1-4d48-8f34-aae4618abe21 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:21.155735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 187/379] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:30:20 2022] GET /identity/v3/projects/6c2fc958121b4494a2ef337a1cb0b9b4 => generated 344 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:21.156989 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c306a-59b1-4d48-8f34-aae4618abe21 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:21.266779 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2745246-87ae-4701-a276-4cba069c15dd None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:21.325861 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1c306a-59b1-4d48-8f34-aae4618abe21 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:21.326091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98cc5d0-b423-419a-9d35-7695f1f399dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:21.326737 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98cc5d0-b423-419a-9d35-7695f1f399dd None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:21.327023 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98cc5d0-b423-419a-9d35-7695f1f399dd None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:21.327311 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98cc5d0-b423-419a-9d35-7695f1f399dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:21.328153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98cc5d0-b423-419a-9d35-7695f1f399dd None admin] RBAC: Adding query filter params (group.id=0dc59d8cece14689b7d92b31c02b0986, role.id=eda4ac05bc364224a7f319e7ded54073, scope.project.id=7abea439f4624f6680796de4a97fc13b) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:21.328465 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98cc5d0-b423-419a-9d35-7695f1f399dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:21.330082 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98cc5d0-b423-419a-9d35-7695f1f399dd None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:21.336794 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 193/380] 10.222.0.13 () {56 vars in 1376 bytes} [Fri May 27 19:30:21 2022] GET /identity/v3/role_assignments?group.id=0dc59d8cece14689b7d92b31c02b0986&scope.project.id=7abea439f4624f6680796de4a97fc13b&role.id=eda4ac05bc364224a7f319e7ded54073 => generated 260 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:21.345488 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98cc5d0-b423-419a-9d35-7695f1f399dd None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:21.373930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e105e37-4e3d-48b2-ad56-71b1a18451fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:21.376339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e105e37-4e3d-48b2-ad56-71b1a18451fe None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:21.377099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e105e37-4e3d-48b2-ad56-71b1a18451fe None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:21.377586 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e105e37-4e3d-48b2-ad56-71b1a18451fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:21.393544 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e105e37-4e3d-48b2-ad56-71b1a18451fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:21.403617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:21.406234 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-9e105e37-4e3d-48b2-ad56-71b1a18451fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 19:30:21.406234 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:21.406234 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:21.406234 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:21.406234 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:21.406234 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:21.406234 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:21.406234 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:21.406234 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:21.406234 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:21.406234 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:21.406234 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:21.406234 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:21.406234 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:21.406234 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:21.406234 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:21.407189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:21.407189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:21.407189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:21.407189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:21.407189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:21.407189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:21.407189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:21.407189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:21.407189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:21.407189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:21.407189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:21.407189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:21.407189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:21.407189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:21.407189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:21.407189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:21.408231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:21.408231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:21.408231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:21.408231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:21.408231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:21.408231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:21.408231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:21.408231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:21.408231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:21.408231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:21.408231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 19:30:21.408231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:21.416486 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 188/381] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:21.419985 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e105e37-4e3d-48b2-ad56-71b1a18451fe None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:21.482522 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:21.547682 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e105e37-4e3d-48b2-ad56-71b1a18451fe None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:21.593053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfebac7-ca28-47b5-83d9-272794bd576a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:21.594127 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfebac7-ca28-47b5-83d9-272794bd576a None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:21.594619 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfebac7-ca28-47b5-83d9-272794bd576a None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:21.595101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfebac7-ca28-47b5-83d9-272794bd576a None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:21.596132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfebac7-ca28-47b5-83d9-272794bd576a None admin] RBAC: Adding query filter params (name=member) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:21.596661 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfebac7-ca28-47b5-83d9-272794bd576a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:21.598327 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfebac7-ca28-47b5-83d9-272794bd576a None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:21.599033 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98cc5d0-b423-419a-9d35-7695f1f399dd None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:21.614583 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 189/382] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:30:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:21.616715 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfebac7-ca28-47b5-83d9-272794bd576a None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:21.700540 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6ce267-76cb-4bf7-8e3e-2bd5a62915a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:21.702137 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ce267-76cb-4bf7-8e3e-2bd5a62915a4 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:21.702433 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ce267-76cb-4bf7-8e3e-2bd5a62915a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:21.702722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6ce267-76cb-4bf7-8e3e-2bd5a62915a4 None admin] PATH_INFO: `/v3/groups/76ad77868bd74a8185ac8c016f4b9411` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:21.711015 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ce267-76cb-4bf7-8e3e-2bd5a62915a4 None admin] RBAC: Authorizing `identity:get_group(group_id=76ad77868bd74a8185ac8c016f4b9411)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:21.712054 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6ce267-76cb-4bf7-8e3e-2bd5a62915a4 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:21.715347 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 194/383] 10.222.0.13 () {56 vars in 1180 bytes} [Fri May 27 19:30:21 2022] GET /identity/v3/groups/76ad77868bd74a8185ac8c016f4b9411 => generated 229 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:21.716535 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719d8e7-bf16-411d-986a-73b68477465c None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:21.716823 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719d8e7-bf16-411d-986a-73b68477465c None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:21.721394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719d8e7-bf16-411d-986a-73b68477465c None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:21.723296 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 195/384] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:21.729228 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75b874-c730-47ed-a540-e8b4aa2551ed None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:21.729546 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75b874-c730-47ed-a540-e8b4aa2551ed None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:21.729797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75b874-c730-47ed-a540-e8b4aa2551ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:21.748170 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfebac7-ca28-47b5-83d9-272794bd576a None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:21.754584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-6b75b874-c730-47ed-a540-e8b4aa2551ed None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:21.756048 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-6b75b874-c730-47ed-a540-e8b4aa2551ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:21.771396 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:21.812378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2276a282-bec7-47c8-8cc6-a9f5801c8d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:21.813492 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276a282-bec7-47c8-8cc6-a9f5801c8d73 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:21.813789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276a282-bec7-47c8-8cc6-a9f5801c8d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:21.814077 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2276a282-bec7-47c8-8cc6-a9f5801c8d73 None admin] PATH_INFO: `/v3/groups/0dc59d8cece14689b7d92b31c02b0986` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:21.815458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2276a282-bec7-47c8-8cc6-a9f5801c8d73 None admin] RBAC: Authorizing `identity:get_group(group_id=0dc59d8cece14689b7d92b31c02b0986)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:21.818556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2276a282-bec7-47c8-8cc6-a9f5801c8d73 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:21.820689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 190/385] 10.222.0.13 () {56 vars in 1180 bytes} [Fri May 27 19:30:21 2022] GET /identity/v3/groups/0dc59d8cece14689b7d92b31c02b0986 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:21.821673 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2276a282-bec7-47c8-8cc6-a9f5801c8d73 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:21.863174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 196/386] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 19:30:21.864411 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75b874-c730-47ed-a540-e8b4aa2551ed None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:21.913346 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:21.957610 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2276a282-bec7-47c8-8cc6-a9f5801c8d73 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:21.997302 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316d8e9-3d16-4f3a-8276-489c36333970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:21.997986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316d8e9-3d16-4f3a-8276-489c36333970 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:21.998272 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316d8e9-3d16-4f3a-8276-489c36333970 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:21.998551 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316d8e9-3d16-4f3a-8276-489c36333970 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:21.999504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4316d8e9-3d16-4f3a-8276-489c36333970 None admin] RBAC: Adding query filter params (group.id=76ad77868bd74a8185ac8c016f4b9411, role.id=6505bbb129404436966ee59aa7a01d8e, scope.project.id=6c2fc958121b4494a2ef337a1cb0b9b4) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:21.999823 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4316d8e9-3d16-4f3a-8276-489c36333970 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:22.001461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4316d8e9-3d16-4f3a-8276-489c36333970 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:22.008054 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 191/387] 10.222.0.13 () {56 vars in 1376 bytes} [Fri May 27 19:30:21 2022] GET /identity/v3/role_assignments?group.id=76ad77868bd74a8185ac8c016f4b9411&scope.project.id=6c2fc958121b4494a2ef337a1cb0b9b4&role.id=6505bbb129404436966ee59aa7a01d8e => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:22.009005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316d8e9-3d16-4f3a-8276-489c36333970 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:22.053493 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75b874-c730-47ed-a540-e8b4aa2551ed None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:22.099730 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea577cda-8267-4578-a469-960010ef9811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:22.100610 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea577cda-8267-4578-a469-960010ef9811 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:22.100895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea577cda-8267-4578-a469-960010ef9811 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:22.101224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea577cda-8267-4578-a469-960010ef9811 None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:22.102740 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea577cda-8267-4578-a469-960010ef9811 None admin] RBAC: Authorizing `identity:get_project(project_id=b30f38a32f6d4982bd829828ccac6fa7)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:22.104867 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea577cda-8267-4578-a469-960010ef9811 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:22.107232 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 197/388] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:30:21 2022] GET /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:22.116676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea577cda-8267-4578-a469-960010ef9811 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:22.132459 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:22.240872 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316d8e9-3d16-4f3a-8276-489c36333970 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:22.294439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6a39e-d4a0-4256-914d-0893359fb8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:22.297457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6a39e-d4a0-4256-914d-0893359fb8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:22.297786 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6a39e-d4a0-4256-914d-0893359fb8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:22.298070 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6a39e-d4a0-4256-914d-0893359fb8f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:22.315625 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd6a39e-d4a0-4256-914d-0893359fb8f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:22.325603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-2dd6a39e-d4a0-4256-914d-0893359fb8f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:30:22.325603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:22.325603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:22.325603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:22.325603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:22.325603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:22.325603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:22.325603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:22.325603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:22.325603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:22.325603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:22.325603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:22.325603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:22.325603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:22.325603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:22.325603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:22.326741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:22.326741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:22.326741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:22.326741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:22.326741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:22.326741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:22.326741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:22.326741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:22.326741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:22.326741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:22.326741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:22.326741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:22.326741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:22.326741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:22.326741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:22.326741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:22.327672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:22.327672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:22.327672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:22.327672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:22.327672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:22.327672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:22.327672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:22.327672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:22.327672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:22.327672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:22.327672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:30:22.327672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:22.327672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 192/389] 10.222.0.13 () {56 vars in 1114 bytes} [Fri May 27 19:30:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:22.332817 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6a39e-d4a0-4256-914d-0893359fb8f5 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:22.337822 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:22.352339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea577cda-8267-4578-a469-960010ef9811 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:22.394179 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091345a4-43f4-47ad-8f3c-8b030d1dd007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:22.395553 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091345a4-43f4-47ad-8f3c-8b030d1dd007 None admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:22.396065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091345a4-43f4-47ad-8f3c-8b030d1dd007 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:22.396656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091345a4-43f4-47ad-8f3c-8b030d1dd007 None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7/groups/0dc59d8cece14689b7d92b31c02b0986/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:22.399520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091345a4-43f4-47ad-8f3c-8b030d1dd007 None admin] RBAC: Authorizing `identity:create_grant(project_id=b30f38a32f6d4982bd829828ccac6fa7, group_id=0dc59d8cece14689b7d92b31c02b0986, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:22.402389 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091345a4-43f4-47ad-8f3c-8b030d1dd007 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:22.467203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6a39e-d4a0-4256-914d-0893359fb8f5 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:22.469091 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:22.514865 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c31c5-cc43-4172-8a0e-e8adf5c200ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:22.514865 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c31c5-cc43-4172-8a0e-e8adf5c200ac None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:22.515097 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c31c5-cc43-4172-8a0e-e8adf5c200ac None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:22.515307 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c31c5-cc43-4172-8a0e-e8adf5c200ac None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:22.516026 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c31c5-cc43-4172-8a0e-e8adf5c200ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:22.516338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c31c5-cc43-4172-8a0e-e8adf5c200ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:22.517849 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c31c5-cc43-4172-8a0e-e8adf5c200ac None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:22.535889 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 193/390] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:30:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:22.541640 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c31c5-cc43-4172-8a0e-e8adf5c200ac None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:22.559732 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 198/391] 10.222.0.13 () {58 vars in 1442 bytes} [Fri May 27 19:30:22 2022] PUT /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7/groups/0dc59d8cece14689b7d92b31c02b0986/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:30:22.671250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c31c5-cc43-4172-8a0e-e8adf5c200ac None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:22.735530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21eff6-a22f-4fae-9ca6-c89529fbdb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:22.736419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21eff6-a22f-4fae-9ca6-c89529fbdb0b None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:22.736699 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21eff6-a22f-4fae-9ca6-c89529fbdb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:22.736990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea21eff6-a22f-4fae-9ca6-c89529fbdb0b None admin] PATH_INFO: `/v3/groups/0dc59d8cece14689b7d92b31c02b0986` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:22.742374 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21eff6-a22f-4fae-9ca6-c89529fbdb0b None admin] RBAC: Authorizing `identity:get_group(group_id=0dc59d8cece14689b7d92b31c02b0986)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:22.743400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea21eff6-a22f-4fae-9ca6-c89529fbdb0b None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:22.746357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 194/392] 10.222.0.13 () {56 vars in 1180 bytes} [Fri May 27 19:30:22 2022] GET /identity/v3/groups/0dc59d8cece14689b7d92b31c02b0986 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:22.752695 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091345a4-43f4-47ad-8f3c-8b030d1dd007 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:22.957560 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091345a4-43f4-47ad-8f3c-8b030d1dd007 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:23.015636 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd27f3-8677-4d5e-93a2-4e28ebf4195a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:23.016561 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd27f3-8677-4d5e-93a2-4e28ebf4195a None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:23.016847 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd27f3-8677-4d5e-93a2-4e28ebf4195a None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:23.017153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd27f3-8677-4d5e-93a2-4e28ebf4195a None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:23.018551 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd27f3-8677-4d5e-93a2-4e28ebf4195a None admin] RBAC: Authorizing `identity:get_project(project_id=b30f38a32f6d4982bd829828ccac6fa7)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:23.020057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd27f3-8677-4d5e-93a2-4e28ebf4195a None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:23.026787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 199/393] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:30:22 2022] GET /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7 => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:23.035089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21eff6-a22f-4fae-9ca6-c89529fbdb0b None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:23.098409 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:23.211984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea21eff6-a22f-4fae-9ca6-c89529fbdb0b None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:23.261332 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18875969-ad90-4915-9097-c1cfad986593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:23.262305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18875969-ad90-4915-9097-c1cfad986593 None admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:23.262599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18875969-ad90-4915-9097-c1cfad986593 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:23.262870 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18875969-ad90-4915-9097-c1cfad986593 None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7/groups/0dc59d8cece14689b7d92b31c02b0986/roles/eda4ac05bc364224a7f319e7ded54073` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:23.265270 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18875969-ad90-4915-9097-c1cfad986593 None admin] RBAC: Authorizing `identity:create_grant(project_id=b30f38a32f6d4982bd829828ccac6fa7, group_id=0dc59d8cece14689b7d92b31c02b0986, role_id=eda4ac05bc364224a7f319e7ded54073)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:23.266968 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18875969-ad90-4915-9097-c1cfad986593 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:23.357940 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 195/394] 10.222.0.13 () {58 vars in 1442 bytes} [Fri May 27 19:30:23 2022] PUT /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7/groups/0dc59d8cece14689b7d92b31c02b0986/roles/eda4ac05bc364224a7f319e7ded54073 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:30:23.789672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90d003-a2ea-4f27-8488-987e12760d85 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:23.790161 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90d003-a2ea-4f27-8488-987e12760d85 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:23.790426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a90d003-a2ea-4f27-8488-987e12760d85 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:23.793536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 200/395] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:23.799773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bbfda-c451-4349-ae2b-cb1b7a6b85e0 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:23.800261 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bbfda-c451-4349-ae2b-cb1b7a6b85e0 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:23.800345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22bbfda-c451-4349-ae2b-cb1b7a6b85e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:23.820799 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-f22bbfda-c451-4349-ae2b-cb1b7a6b85e0 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:23.822520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-f22bbfda-c451-4349-ae2b-cb1b7a6b85e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:23.917897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 196/396] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:23.958935 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90d003-a2ea-4f27-8488-987e12760d85 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:23.979112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:24.102924 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a90d003-a2ea-4f27-8488-987e12760d85 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:24.155659 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e5aaf-bb6d-45b5-9c34-edf024b6f60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:24.156490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e5aaf-bb6d-45b5-9c34-edf024b6f60e None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:24.156773 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e5aaf-bb6d-45b5-9c34-edf024b6f60e None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:24.157313 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e5aaf-bb6d-45b5-9c34-edf024b6f60e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:24.176480 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e5aaf-bb6d-45b5-9c34-edf024b6f60e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:24.185459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-4c2e5aaf-bb6d-45b5-9c34-edf024b6f60e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 19:30:24.185459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:24.185459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:24.185459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:24.185459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:24.185459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:24.185459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:24.185459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:24.185459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:24.185459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:24.185459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:24.185459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:24.185459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:24.185459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:24.185459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:24.185459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:24.186436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:24.186436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:24.186436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:24.186436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:24.186436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:24.186436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:24.186436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:24.186436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:24.186436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:24.186436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:24.186436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:24.186436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:24.186436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:24.186436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:24.186436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:24.186436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:24.187248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:24.187248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:24.187248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:24.187248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:24.187248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:24.187248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:24.187248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:24.187248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:24.187248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:24.187248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:24.187248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 19:30:24.187248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:24.188422 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 201/397] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:24.193117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22bbfda-c451-4349-ae2b-cb1b7a6b85e0 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:24.225143 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:24.335125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22bbfda-c451-4349-ae2b-cb1b7a6b85e0 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:24.390769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ceac8-fea9-47cd-8739-9561424dd6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:24.391655 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ceac8-fea9-47cd-8739-9561424dd6a8 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:24.391938 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ceac8-fea9-47cd-8739-9561424dd6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:24.392217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ceac8-fea9-47cd-8739-9561424dd6a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:24.392990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ceac8-fea9-47cd-8739-9561424dd6a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:24.393303 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ceac8-fea9-47cd-8739-9561424dd6a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:24.394672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ceac8-fea9-47cd-8739-9561424dd6a8 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:24.410047 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 197/398] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:30:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:24.416123 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e5aaf-bb6d-45b5-9c34-edf024b6f60e None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:24.503153 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe782d-4084-4b2b-93d5-2c85fef40bce None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:24.503531 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe782d-4084-4b2b-93d5-2c85fef40bce None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:24.503690 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe782d-4084-4b2b-93d5-2c85fef40bce None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:24.505157 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 198/399] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 19:30:24.515980 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346966b2-31af-4051-b00b-19c6e83f6f2b None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:24.515980 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346966b2-31af-4051-b00b-19c6e83f6f2b None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:24.515980 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346966b2-31af-4051-b00b-19c6e83f6f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:24.562016 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-346966b2-31af-4051-b00b-19c6e83f6f2b None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:24.563208 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-346966b2-31af-4051-b00b-19c6e83f6f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:24.580092 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:24.628390 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e5aaf-bb6d-45b5-9c34-edf024b6f60e None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:24.702752 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 199/400] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:24.722772 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221975b-d2c9-4763-8d2f-0974f2eb0aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:24.724008 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221975b-d2c9-4763-8d2f-0974f2eb0aca None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:24.724500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221975b-d2c9-4763-8d2f-0974f2eb0aca None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:24.724981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221975b-d2c9-4763-8d2f-0974f2eb0aca None admin] PATH_INFO: `/v3/groups/0dc59d8cece14689b7d92b31c02b0986` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:24.730188 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221975b-d2c9-4763-8d2f-0974f2eb0aca None admin] RBAC: Authorizing `identity:get_group(group_id=0dc59d8cece14689b7d92b31c02b0986)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:24.732020 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221975b-d2c9-4763-8d2f-0974f2eb0aca None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:24.733738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 202/401] 10.222.0.13 () {56 vars in 1180 bytes} [Fri May 27 19:30:24 2022] GET /identity/v3/groups/0dc59d8cece14689b7d92b31c02b0986 => generated 226 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:24.739599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346966b2-31af-4051-b00b-19c6e83f6f2b None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:24.749353 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221975b-d2c9-4763-8d2f-0974f2eb0aca None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:24.764283 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:24.885540 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221975b-d2c9-4763-8d2f-0974f2eb0aca None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:24.904747 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346966b2-31af-4051-b00b-19c6e83f6f2b None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:24.937117 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a23ba-e646-45fa-b9fe-b78b7eff8afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:24.938461 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a23ba-e646-45fa-b9fe-b78b7eff8afe None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:24.939079 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a23ba-e646-45fa-b9fe-b78b7eff8afe None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:24.939578 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33a23ba-e646-45fa-b9fe-b78b7eff8afe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:24.957097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33a23ba-e646-45fa-b9fe-b78b7eff8afe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:24.966163 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d7f93-546e-4c0e-a42f-e21edfbc30a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:24.967046 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d7f93-546e-4c0e-a42f-e21edfbc30a3 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:24.967351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d7f93-546e-4c0e-a42f-e21edfbc30a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:24.967629 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d7f93-546e-4c0e-a42f-e21edfbc30a3 None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:24.968853 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d7f93-546e-4c0e-a42f-e21edfbc30a3 None admin] RBAC: Authorizing `identity:get_project(project_id=7abea439f4624f6680796de4a97fc13b)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:24.970349 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d7f93-546e-4c0e-a42f-e21edfbc30a3 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:24.976151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 200/402] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:30:24 2022] GET /identity/v3/projects/7abea439f4624f6680796de4a97fc13b => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:24.976151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bf46c-eb96-476f-a03b-184f30044080 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:24.976151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bf46c-eb96-476f-a03b-184f30044080 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:24.976151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4bf46c-eb96-476f-a03b-184f30044080 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:24.976151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 201/403] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:24.978902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-e33a23ba-e646-45fa-b9fe-b78b7eff8afe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:30:24.978902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:24.978902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:24.978902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:24.978902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:24.978902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:24.978902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:24.978902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:24.978902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:24.978902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:24.978902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:24.978902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:24.978902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:24.978902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:24.978902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:24.978902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:24.980681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:24.980681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:24.980681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:24.980681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:24.980681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:24.980681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:24.980681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:24.980681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:24.980681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:24.980681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:24.980681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:24.980681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:24.980681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:24.980681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:24.980681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:24.980681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:24.981670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:24.981670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:24.981670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:24.981670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:24.981670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:24.981670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:24.981670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:24.981670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:24.981670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:24.981670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:24.981670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:30:24.981670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:24.984162 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2930c-8e5b-4fc1-9e76-fd74970a82f6 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:24.984679 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2930c-8e5b-4fc1-9e76-fd74970a82f6 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:24.985210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2930c-8e5b-4fc1-9e76-fd74970a82f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:24.986760 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 203/404] 10.222.0.13 () {56 vars in 1114 bytes} [Fri May 27 19:30:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:24.988170 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a23ba-e646-45fa-b9fe-b78b7eff8afe None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:24.994784 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:25.011153 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-6cc2930c-8e5b-4fc1-9e76-fd74970a82f6 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:25.012903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-6cc2930c-8e5b-4fc1-9e76-fd74970a82f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:25.117987 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 202/405] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:25.119973 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2930c-8e5b-4fc1-9e76-fd74970a82f6 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:25.137264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:25.152593 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33a23ba-e646-45fa-b9fe-b78b7eff8afe None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:25.204071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81f0fa-dc6a-4d9c-a387-b409468348ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:25.205674 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81f0fa-dc6a-4d9c-a387-b409468348ae None admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:25.206343 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81f0fa-dc6a-4d9c-a387-b409468348ae None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:25.206822 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81f0fa-dc6a-4d9c-a387-b409468348ae None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b/groups/0dc59d8cece14689b7d92b31c02b0986/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:25.209687 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81f0fa-dc6a-4d9c-a387-b409468348ae None admin] RBAC: Authorizing `identity:create_grant(project_id=7abea439f4624f6680796de4a97fc13b, group_id=0dc59d8cece14689b7d92b31c02b0986, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:25.213050 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81f0fa-dc6a-4d9c-a387-b409468348ae None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:25.269347 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc2930c-8e5b-4fc1-9e76-fd74970a82f6 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:25.320648 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b2ab0-7030-49f0-9af8-587534b0cf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:25.321371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b2ab0-7030-49f0-9af8-587534b0cf19 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:25.321695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b2ab0-7030-49f0-9af8-587534b0cf19 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:25.322144 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b2ab0-7030-49f0-9af8-587534b0cf19 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:25.323194 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b2ab0-7030-49f0-9af8-587534b0cf19 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:25.323647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b2ab0-7030-49f0-9af8-587534b0cf19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:25.325841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86b2ab0-7030-49f0-9af8-587534b0cf19 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:25.345735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 203/407] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:30:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:25.345735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b2ab0-7030-49f0-9af8-587534b0cf19 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:25.346430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 204/407] 10.222.0.13 () {58 vars in 1442 bytes} [Fri May 27 19:30:24 2022] PUT /identity/v3/projects/7abea439f4624f6680796de4a97fc13b/groups/0dc59d8cece14689b7d92b31c02b0986/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:30:25.348213 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81f0fa-dc6a-4d9c-a387-b409468348ae None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:25.500769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:25.531356 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:25.565056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86b2ab0-7030-49f0-9af8-587534b0cf19 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:25.574735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81f0fa-dc6a-4d9c-a387-b409468348ae None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:25.608893 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff82a53-46cb-4e66-b144-ff36c2edea8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:25.609742 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff82a53-46cb-4e66-b144-ff36c2edea8f None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:25.610029 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff82a53-46cb-4e66-b144-ff36c2edea8f None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:25.611279 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff82a53-46cb-4e66-b144-ff36c2edea8f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:25.628832 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff82a53-46cb-4e66-b144-ff36c2edea8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:25.629898 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff48e10-818a-4b60-927f-36e2e5c5b1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:25.630781 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff48e10-818a-4b60-927f-36e2e5c5b1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:25.631072 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff48e10-818a-4b60-927f-36e2e5c5b1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:25.631478 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff48e10-818a-4b60-927f-36e2e5c5b1a3 None admin] PATH_INFO: `/v3/groups/0dc59d8cece14689b7d92b31c02b0986` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:25.633148 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff48e10-818a-4b60-927f-36e2e5c5b1a3 None admin] RBAC: Authorizing `identity:get_group(group_id=0dc59d8cece14689b7d92b31c02b0986)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:25.634988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff48e10-818a-4b60-927f-36e2e5c5b1a3 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:25.636712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-dff82a53-46cb-4e66-b144-ff36c2edea8f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:30:25.636712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:25.636712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:25.636712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:25.636712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:25.636712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:25.636712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:25.636712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:25.636712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:25.636712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:25.636712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:25.636712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:25.636712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:25.636712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:25.636712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:25.636712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:25.637623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:25.637623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:25.637623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:25.637623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:25.637623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:25.637623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:25.637623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:25.637623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:25.637623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:25.637623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:25.637623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:25.637623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:25.637623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:25.637623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:25.637623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:25.637623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:25.639657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:25.639657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:25.639657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:25.639657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:25.639657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:25.639657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:25.639657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:25.639657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:25.639657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:25.639657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:25.639657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:30:25.639657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:25.640338 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 205/408] 10.222.0.13 () {56 vars in 1180 bytes} [Fri May 27 19:30:25 2022] GET /identity/v3/groups/0dc59d8cece14689b7d92b31c02b0986 => generated 226 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:25.640411 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 204/409] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:30:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:25.643782 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff82a53-46cb-4e66-b144-ff36c2edea8f None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:25.646361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07e3be-16b8-48b4-8288-ac99b83e95f8 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:25.646662 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07e3be-16b8-48b4-8288-ac99b83e95f8 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:25.646933 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07e3be-16b8-48b4-8288-ac99b83e95f8 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:25.650794 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 206/410] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:25 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:25.650965 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07e3be-16b8-48b4-8288-ac99b83e95f8 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:25.787363 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff82a53-46cb-4e66-b144-ff36c2edea8f None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:25.790027 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07e3be-16b8-48b4-8288-ac99b83e95f8 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:25.825212 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:25.829762 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eef672-52f8-487d-ae2f-b553601e5df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:25.830799 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eef672-52f8-487d-ae2f-b553601e5df4 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:25.835446 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eef672-52f8-487d-ae2f-b553601e5df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:25.835446 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eef672-52f8-487d-ae2f-b553601e5df4 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:25.835610 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974261d4-7596-478f-bc96-3b2fd202c350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:25.835717 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eef672-52f8-487d-ae2f-b553601e5df4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:25.835717 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eef672-52f8-487d-ae2f-b553601e5df4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:25.835842 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974261d4-7596-478f-bc96-3b2fd202c350 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:25.835842 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974261d4-7596-478f-bc96-3b2fd202c350 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:25.835842 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974261d4-7596-478f-bc96-3b2fd202c350 None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:25.836019 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eef672-52f8-487d-ae2f-b553601e5df4 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:25.836086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974261d4-7596-478f-bc96-3b2fd202c350 None admin] RBAC: Authorizing `identity:get_project(project_id=7abea439f4624f6680796de4a97fc13b)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:25.836361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974261d4-7596-478f-bc96-3b2fd202c350 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:25.836875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:25.839655 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 207/411] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:30:25 2022] GET /identity/v3/projects/7abea439f4624f6680796de4a97fc13b => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:25.841096 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cd7c6-d163-4fc8-baa0-bd440ea92e5a None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:25.841475 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cd7c6-d163-4fc8-baa0-bd440ea92e5a None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:25.841830 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48cd7c6-d163-4fc8-baa0-bd440ea92e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:25.848139 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 205/412] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:25.854967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eef672-52f8-487d-ae2f-b553601e5df4 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:25.864889 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-c48cd7c6-d163-4fc8-baa0-bd440ea92e5a None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:25.870636 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-c48cd7c6-d163-4fc8-baa0-bd440ea92e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:25.940825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 208/413] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:25.942599 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cd7c6-d163-4fc8-baa0-bd440ea92e5a None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:25.970941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eef672-52f8-487d-ae2f-b553601e5df4 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:26.012198 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c99376-1da2-4278-8438-2fe992ccb4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:26.013479 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c99376-1da2-4278-8438-2fe992ccb4bf None admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:26.013959 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c99376-1da2-4278-8438-2fe992ccb4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:26.014502 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c99376-1da2-4278-8438-2fe992ccb4bf None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b/groups/0dc59d8cece14689b7d92b31c02b0986/roles/eda4ac05bc364224a7f319e7ded54073` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:26.018395 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c99376-1da2-4278-8438-2fe992ccb4bf None admin] RBAC: Authorizing `identity:create_grant(project_id=7abea439f4624f6680796de4a97fc13b, group_id=0dc59d8cece14689b7d92b31c02b0986, role_id=eda4ac05bc364224a7f319e7ded54073)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:26.021229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c99376-1da2-4278-8438-2fe992ccb4bf None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:26.052595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48cd7c6-d163-4fc8-baa0-bd440ea92e5a None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:26.091442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:26.101193 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 206/414] 10.222.0.13 () {58 vars in 1442 bytes} [Fri May 27 19:30:25 2022] PUT /identity/v3/projects/7abea439f4624f6680796de4a97fc13b/groups/0dc59d8cece14689b7d92b31c02b0986/roles/eda4ac05bc364224a7f319e7ded54073 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:30:26.104636 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c99376-1da2-4278-8438-2fe992ccb4bf None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:26.111646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c12c04-ed26-4394-a658-6263e3e8dc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:26.114445 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c12c04-ed26-4394-a658-6263e3e8dc02 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:26.114445 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c12c04-ed26-4394-a658-6263e3e8dc02 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:26.114445 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c12c04-ed26-4394-a658-6263e3e8dc02 None admin] PATH_INFO: `/v3/groups/76ad77868bd74a8185ac8c016f4b9411` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:26.119952 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c12c04-ed26-4394-a658-6263e3e8dc02 None admin] RBAC: Authorizing `identity:get_group(group_id=76ad77868bd74a8185ac8c016f4b9411)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:26.123419 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c12c04-ed26-4394-a658-6263e3e8dc02 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:26.128661 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 209/415] 10.222.0.13 () {56 vars in 1180 bytes} [Fri May 27 19:30:25 2022] GET /identity/v3/groups/76ad77868bd74a8185ac8c016f4b9411 => generated 229 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:26.133353 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c12c04-ed26-4394-a658-6263e3e8dc02 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:26.252415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:26.288983 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c12c04-ed26-4394-a658-6263e3e8dc02 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:26.332928 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876198d-a7c5-43c9-b4c6-7ea6e5f26f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:26.334340 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876198d-a7c5-43c9-b4c6-7ea6e5f26f04 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:26.334624 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876198d-a7c5-43c9-b4c6-7ea6e5f26f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:26.334903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5876198d-a7c5-43c9-b4c6-7ea6e5f26f04 None admin] PATH_INFO: `/v3/projects/6c2fc958121b4494a2ef337a1cb0b9b4` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:26.340145 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5876198d-a7c5-43c9-b4c6-7ea6e5f26f04 None admin] RBAC: Authorizing `identity:get_project(project_id=6c2fc958121b4494a2ef337a1cb0b9b4)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:26.341792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5876198d-a7c5-43c9-b4c6-7ea6e5f26f04 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:26.345765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 210/416] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:30:26 2022] GET /identity/v3/projects/6c2fc958121b4494a2ef337a1cb0b9b4 => generated 344 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:26.353223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876198d-a7c5-43c9-b4c6-7ea6e5f26f04 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:26.396058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c99376-1da2-4278-8438-2fe992ccb4bf None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:26.427571 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:26.445714 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a78309-7d57-47b3-b680-7d21a4b66bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:26.447225 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a78309-7d57-47b3-b680-7d21a4b66bcd None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:26.448019 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a78309-7d57-47b3-b680-7d21a4b66bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:26.448300 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a78309-7d57-47b3-b680-7d21a4b66bcd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:26.465536 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a78309-7d57-47b3-b680-7d21a4b66bcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:26.473127 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5876198d-a7c5-43c9-b4c6-7ea6e5f26f04 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:26.475392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-b9a78309-7d57-47b3-b680-7d21a4b66bcd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 19:30:26.475392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:26.475392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:26.475392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:26.475392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:26.475392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:26.475392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:26.475392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:26.475392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:26.475392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:26.475392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:26.475392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:26.475392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:26.475392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:26.475392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:26.475392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:26.476592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:26.476592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:26.476592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:26.476592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:26.476592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:26.476592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:26.476592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:26.476592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:26.476592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:26.476592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:26.476592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:26.476592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:26.476592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:26.476592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:26.476592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:26.476592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:26.477520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:26.477520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:26.477520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:26.477520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:26.477520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:26.477520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:26.477520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:26.477520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:26.477520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:26.477520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:26.477520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 19:30:26.477520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:26.478484 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 207/417] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:26.484848 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a78309-7d57-47b3-b680-7d21a4b66bcd None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:26.516465 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d79fe-9e2b-46c7-bf87-76d2920212d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:26.517738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d79fe-9e2b-46c7-bf87-76d2920212d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:26.518259 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d79fe-9e2b-46c7-bf87-76d2920212d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:26.518725 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d79fe-9e2b-46c7-bf87-76d2920212d0 None admin] PATH_INFO: `/v3/projects/6c2fc958121b4494a2ef337a1cb0b9b4/groups/76ad77868bd74a8185ac8c016f4b9411/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:26.528141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d79fe-9e2b-46c7-bf87-76d2920212d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c2fc958121b4494a2ef337a1cb0b9b4, group_id=76ad77868bd74a8185ac8c016f4b9411, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:26.530276 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d79fe-9e2b-46c7-bf87-76d2920212d0 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:26.615094 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a78309-7d57-47b3-b680-7d21a4b66bcd None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:26.629286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 211/418] 10.222.0.13 () {58 vars in 1442 bytes} [Fri May 27 19:30:26 2022] PUT /identity/v3/projects/6c2fc958121b4494a2ef337a1cb0b9b4/groups/76ad77868bd74a8185ac8c016f4b9411/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:30:26.631534 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9c184-11b4-435a-b5f0-b9daed154b71 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:26.632013 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9c184-11b4-435a-b5f0-b9daed154b71 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:26.634828 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9c184-11b4-435a-b5f0-b9daed154b71 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:26.636455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 212/419] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:26.640752 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:26.652596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32c5b0-f372-4ca2-bdd3-922362be8978 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:26.653233 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32c5b0-f372-4ca2-bdd3-922362be8978 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:26.653680 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32c5b0-f372-4ca2-bdd3-922362be8978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:26.662181 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b4e92-661a-4d7e-a3a4-b0ec86e3176e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:26.662875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b4e92-661a-4d7e-a3a4-b0ec86e3176e None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:26.664414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b4e92-661a-4d7e-a3a4-b0ec86e3176e None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:26.664414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b4e92-661a-4d7e-a3a4-b0ec86e3176e None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:26.664414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b4e92-661a-4d7e-a3a4-b0ec86e3176e None admin] RBAC: Adding query filter params (name=member) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:26.664836 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b4e92-661a-4d7e-a3a4-b0ec86e3176e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:26.666452 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b4e92-661a-4d7e-a3a4-b0ec86e3176e None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:26.677177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 208/420] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:30:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:26.689675 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b4e92-661a-4d7e-a3a4-b0ec86e3176e None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:26.706392 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-dc32c5b0-f372-4ca2-bdd3-922362be8978 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:26.707694 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-dc32c5b0-f372-4ca2-bdd3-922362be8978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:26.810686 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b4e92-661a-4d7e-a3a4-b0ec86e3176e None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:26.855476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5558c-402a-4a6d-a938-5d52077fd805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:26.856286 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5558c-402a-4a6d-a938-5d52077fd805 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:26.856576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5558c-402a-4a6d-a938-5d52077fd805 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:26.856857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5558c-402a-4a6d-a938-5d52077fd805 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:26.917803 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 213/421] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 19:30:26.933424 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:26.948761 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32c5b0-f372-4ca2-bdd3-922362be8978 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:26.968733 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e5558c-402a-4a6d-a938-5d52077fd805 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:26.970969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e5558c-402a-4a6d-a938-5d52077fd805 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:27.028146 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:27.092980 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32c5b0-f372-4ca2-bdd3-922362be8978 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:27.161689 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e8493a-d567-4c36-8515-756c6fd7e5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:27.162513 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8493a-d567-4c36-8515-756c6fd7e5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:27.162798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8493a-d567-4c36-8515-756c6fd7e5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:27.163077 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e8493a-d567-4c36-8515-756c6fd7e5b8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:27.178648 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e8493a-d567-4c36-8515-756c6fd7e5b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:27.187865 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 209/422] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:30:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 19:30:27.188287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-c4e8493a-d567-4c36-8515-756c6fd7e5b8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:30:27.188287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:27.188287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:27.188287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:27.188287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:27.188287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:27.188287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:27.188287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:27.188287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:27.188287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:27.188287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:27.188287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:27.188287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:27.188287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:27.188287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:27.188287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:27.189722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:27.189722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:27.189722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:27.189722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:27.189722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:27.189722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:27.189722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:27.189722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:27.189722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:27.189722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:27.189722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:27.189722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:27.189722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:27.189722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:27.189722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:27.189722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:27.190997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:27.190997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:27.190997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:27.190997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:27.190997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:27.190997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:27.190997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:27.190997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:27.190997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:27.190997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:27.190997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:30:27.190997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:27.190997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 214/423] 10.222.0.13 () {56 vars in 1114 bytes} [Fri May 27 19:30:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:27.198059 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5558c-402a-4a6d-a938-5d52077fd805 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:27.198536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e8493a-d567-4c36-8515-756c6fd7e5b8 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:27.348158 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5558c-402a-4a6d-a938-5d52077fd805 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:27.385538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f560b-bcf3-40a4-b612-5611991c0077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:27.386419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f560b-bcf3-40a4-b612-5611991c0077 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:27.386705 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f560b-bcf3-40a4-b612-5611991c0077 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:27.386985 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f560b-bcf3-40a4-b612-5611991c0077 None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:27.388189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f560b-bcf3-40a4-b612-5611991c0077 None admin] RBAC: Authorizing `identity:get_project(project_id=b30f38a32f6d4982bd829828ccac6fa7)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:27.389694 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f560b-bcf3-40a4-b612-5611991c0077 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:27.391757 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 210/424] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:30:27 2022] GET /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:27.393376 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:27.404479 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f560b-bcf3-40a4-b612-5611991c0077 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:27.431377 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e8493a-d567-4c36-8515-756c6fd7e5b8 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:27.461319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:27.508754 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ac1f0-c8a8-4814-8c37-78a3bcf207c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:27.509462 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ac1f0-c8a8-4814-8c37-78a3bcf207c1 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:27.509745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ac1f0-c8a8-4814-8c37-78a3bcf207c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:27.510024 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ac1f0-c8a8-4814-8c37-78a3bcf207c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:27.510780 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ac1f0-c8a8-4814-8c37-78a3bcf207c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:27.511086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ac1f0-c8a8-4814-8c37-78a3bcf207c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:27.516974 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ac1f0-c8a8-4814-8c37-78a3bcf207c1 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:27.531146 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 215/425] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:30:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:27.542478 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ac1f0-c8a8-4814-8c37-78a3bcf207c1 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:27.574405 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f560b-bcf3-40a4-b612-5611991c0077 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:27.610440 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46229c6-30bf-4f95-9b77-6f97d87dec89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:27.611306 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46229c6-30bf-4f95-9b77-6f97d87dec89 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:27.611589 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46229c6-30bf-4f95-9b77-6f97d87dec89 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:27.611866 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46229c6-30bf-4f95-9b77-6f97d87dec89 None admin] PATH_INFO: `/v3/groups/0dc59d8cece14689b7d92b31c02b0986` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:27.613112 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46229c6-30bf-4f95-9b77-6f97d87dec89 None admin] RBAC: Authorizing `identity:get_group(group_id=0dc59d8cece14689b7d92b31c02b0986)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:27.614648 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46229c6-30bf-4f95-9b77-6f97d87dec89 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:27.617210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 211/426] 10.222.0.13 () {56 vars in 1180 bytes} [Fri May 27 19:30:27 2022] GET /identity/v3/groups/0dc59d8cece14689b7d92b31c02b0986 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:27.625611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46229c6-30bf-4f95-9b77-6f97d87dec89 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:27.669351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:27.757375 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ac1f0-c8a8-4814-8c37-78a3bcf207c1 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:27.793634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f448a5ff-f365-47cd-8431-1b886b71a7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:27.794311 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448a5ff-f365-47cd-8431-1b886b71a7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:27.794603 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448a5ff-f365-47cd-8431-1b886b71a7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:27.794956 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f448a5ff-f365-47cd-8431-1b886b71a7b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:27.865065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46229c6-30bf-4f95-9b77-6f97d87dec89 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:27.887931 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:27.937677 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f448a5ff-f365-47cd-8431-1b886b71a7b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:27.939896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f448a5ff-f365-47cd-8431-1b886b71a7b9 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:27.944126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711225a-9cb3-4916-bd1c-3d8e3af2e4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:27.944800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711225a-9cb3-4916-bd1c-3d8e3af2e4a3 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:27.945109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711225a-9cb3-4916-bd1c-3d8e3af2e4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:27.945388 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711225a-9cb3-4916-bd1c-3d8e3af2e4a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:27.946977 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711225a-9cb3-4916-bd1c-3d8e3af2e4a3 None admin] RBAC: Adding query filter params (group.id=0dc59d8cece14689b7d92b31c02b0986, role.id=cb43a575e47147c799eec2f362f4fe0e, scope.project.id=b30f38a32f6d4982bd829828ccac6fa7) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:27.949541 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711225a-9cb3-4916-bd1c-3d8e3af2e4a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:27.951161 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711225a-9cb3-4916-bd1c-3d8e3af2e4a3 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:27.968227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 212/427] 10.222.0.13 () {56 vars in 1376 bytes} [Fri May 27 19:30:27 2022] GET /identity/v3/role_assignments?group.id=0dc59d8cece14689b7d92b31c02b0986&scope.project.id=b30f38a32f6d4982bd829828ccac6fa7&role.id=cb43a575e47147c799eec2f362f4fe0e => generated 611 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:28.081753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 216/428] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:30:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:30:28.089730 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711225a-9cb3-4916-bd1c-3d8e3af2e4a3 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:28.156884 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226b4b0-6f88-40a2-9a8d-f741070be120 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:28.157241 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226b4b0-6f88-40a2-9a8d-f741070be120 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:28.157495 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d226b4b0-6f88-40a2-9a8d-f741070be120 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:28.159194 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 217/429] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 19:30:28.166520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d0031-ec80-45c1-8c20-a3b2b0c244d2 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:28.166939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d0031-ec80-45c1-8c20-a3b2b0c244d2 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:28.167223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54d0031-ec80-45c1-8c20-a3b2b0c244d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:28.197167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-e54d0031-ec80-45c1-8c20-a3b2b0c244d2 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:28.198802 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-e54d0031-ec80-45c1-8c20-a3b2b0c244d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:28.244190 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:28.267231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711225a-9cb3-4916-bd1c-3d8e3af2e4a3 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:28.299231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 218/430] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 19:30:28.315176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf00b2-8282-4178-b38f-331f0d30bd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:28.316111 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf00b2-8282-4178-b38f-331f0d30bd19 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:28.316397 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf00b2-8282-4178-b38f-331f0d30bd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:28.316723 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbf00b2-8282-4178-b38f-331f0d30bd19 None admin] PATH_INFO: `/v3/projects/b30f38a32f6d4982bd829828ccac6fa7` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:28.318055 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbf00b2-8282-4178-b38f-331f0d30bd19 None admin] RBAC: Authorizing `identity:get_project(project_id=b30f38a32f6d4982bd829828ccac6fa7)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:28.319789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbf00b2-8282-4178-b38f-331f0d30bd19 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:28.322112 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 213/431] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:30:28 2022] GET /identity/v3/projects/b30f38a32f6d4982bd829828ccac6fa7 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:28.327399 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d0031-ec80-45c1-8c20-a3b2b0c244d2 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:28.333002 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf00b2-8282-4178-b38f-331f0d30bd19 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:28.339380 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:28.445825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54d0031-ec80-45c1-8c20-a3b2b0c244d2 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:28.450931 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbf00b2-8282-4178-b38f-331f0d30bd19 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:28.486479 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f67fa-ad3c-4f20-9cd7-360821934115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:28.487697 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f67fa-ad3c-4f20-9cd7-360821934115 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:28.488157 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f67fa-ad3c-4f20-9cd7-360821934115 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:28.488641 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f67fa-ad3c-4f20-9cd7-360821934115 None admin] PATH_INFO: `/v3/groups/0dc59d8cece14689b7d92b31c02b0986` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:28.490093 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f67fa-ad3c-4f20-9cd7-360821934115 None admin] RBAC: Authorizing `identity:get_group(group_id=0dc59d8cece14689b7d92b31c02b0986)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:28.491782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f67fa-ad3c-4f20-9cd7-360821934115 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:28.494143 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 219/432] 10.222.0.13 () {56 vars in 1180 bytes} [Fri May 27 19:30:28 2022] GET /identity/v3/groups/0dc59d8cece14689b7d92b31c02b0986 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:28.495441 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3919bdf-d70f-4054-ad6d-6e0a85afea47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:28.496333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3919bdf-d70f-4054-ad6d-6e0a85afea47 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:28.496628 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3919bdf-d70f-4054-ad6d-6e0a85afea47 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:28.496905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3919bdf-d70f-4054-ad6d-6e0a85afea47 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:28.499654 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f67fa-ad3c-4f20-9cd7-360821934115 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:28.517770 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3919bdf-d70f-4054-ad6d-6e0a85afea47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:28.531132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-f3919bdf-d70f-4054-ad6d-6e0a85afea47 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 19:30:28.531132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:28.531132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:28.531132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:28.531132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:28.531132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:28.531132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:28.531132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:28.531132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:28.531132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:28.531132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:28.531132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:28.531132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:28.531132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:28.531132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:28.531132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:28.532088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:28.532088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:28.532088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:28.532088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:28.532088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:28.532088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:28.532088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:28.532088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:28.532088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:28.532088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:28.532088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:28.532088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:28.532088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:28.532088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:28.532088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:28.532088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:28.532897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:28.532897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:28.532897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:28.532897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:28.532897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:28.532897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:28.532897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:28.532897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:28.532897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:28.532897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:28.532897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 19:30:28.532897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:28.534950 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 214/433] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:28.541718 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3919bdf-d70f-4054-ad6d-6e0a85afea47 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:28.553171 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:28.615774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:28.660795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f67fa-ad3c-4f20-9cd7-360821934115 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:28.667868 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3919bdf-d70f-4054-ad6d-6e0a85afea47 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:28.717823 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26558e-fe3e-4e7e-ab87-ca0a99edbd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:28.718518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26558e-fe3e-4e7e-ab87-ca0a99edbd93 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:28.718793 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26558e-fe3e-4e7e-ab87-ca0a99edbd93 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:28.719068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26558e-fe3e-4e7e-ab87-ca0a99edbd93 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:28.719862 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26558e-fe3e-4e7e-ab87-ca0a99edbd93 None admin] RBAC: Adding query filter params (name=member) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:28.720155 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26558e-fe3e-4e7e-ab87-ca0a99edbd93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:28.721879 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26558e-fe3e-4e7e-ab87-ca0a99edbd93 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:28.733963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 215/434] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:30:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:28.737641 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a86c78-93ea-45b6-a502-778186975448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:28.739522 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a86c78-93ea-45b6-a502-778186975448 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:28.739679 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26558e-fe3e-4e7e-ab87-ca0a99edbd93 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:28.740455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a86c78-93ea-45b6-a502-778186975448 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:28.741262 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a86c78-93ea-45b6-a502-778186975448 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:28.743315 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a86c78-93ea-45b6-a502-778186975448 None admin] RBAC: Adding query filter params (group.id=0dc59d8cece14689b7d92b31c02b0986, role.id=eda4ac05bc364224a7f319e7ded54073, scope.project.id=b30f38a32f6d4982bd829828ccac6fa7) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:28.744183 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a86c78-93ea-45b6-a502-778186975448 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:28.747237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a86c78-93ea-45b6-a502-778186975448 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:28.763707 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 220/435] 10.222.0.13 () {56 vars in 1376 bytes} [Fri May 27 19:30:28 2022] GET /identity/v3/role_assignments?group.id=0dc59d8cece14689b7d92b31c02b0986&scope.project.id=b30f38a32f6d4982bd829828ccac6fa7&role.id=eda4ac05bc364224a7f319e7ded54073 => generated 611 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:28.828105 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:28.870180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26558e-fe3e-4e7e-ab87-ca0a99edbd93 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:28.910398 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e1712b-e88e-4eb9-8e15-47a51afe5bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:28.911051 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1712b-e88e-4eb9-8e15-47a51afe5bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:28.911346 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1712b-e88e-4eb9-8e15-47a51afe5bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:28.911648 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e1712b-e88e-4eb9-8e15-47a51afe5bf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:28.997910 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1712b-e88e-4eb9-8e15-47a51afe5bf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:28.999422 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e1712b-e88e-4eb9-8e15-47a51afe5bf8 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:29.085235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:29.131132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 216/436] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:30:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:30:29.136542 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a86c78-93ea-45b6-a502-778186975448 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:29.246295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90b132-f346-4283-927a-428c816fb6df None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:29.246613 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90b132-f346-4283-927a-428c816fb6df None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:29.246971 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd90b132-f346-4283-927a-428c816fb6df None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:29.248357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 217/437] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 19:30:29.254316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3658054-cdfe-445c-957c-64a7740ce919 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:29.254609 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3658054-cdfe-445c-957c-64a7740ce919 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:29.255010 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3658054-cdfe-445c-957c-64a7740ce919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:29.259131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a86c78-93ea-45b6-a502-778186975448 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:29.278492 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-d3658054-cdfe-445c-957c-64a7740ce919 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:29.279694 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d3658054-cdfe-445c-957c-64a7740ce919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:29.296888 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fd654-1497-4c81-ad32-d120b1e69be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:29.297801 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fd654-1497-4c81-ad32-d120b1e69be8 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:29.298085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fd654-1497-4c81-ad32-d120b1e69be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:29.298362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209fd654-1497-4c81-ad32-d120b1e69be8 None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:29.299548 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209fd654-1497-4c81-ad32-d120b1e69be8 None admin] RBAC: Authorizing `identity:get_project(project_id=7abea439f4624f6680796de4a97fc13b)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:29.304096 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209fd654-1497-4c81-ad32-d120b1e69be8 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:29.304096 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 221/438] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:30:29 2022] GET /identity/v3/projects/7abea439f4624f6680796de4a97fc13b => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:29.308814 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fd654-1497-4c81-ad32-d120b1e69be8 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:29.323229 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:29.376534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 218/439] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:29.401385 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3658054-cdfe-445c-957c-64a7740ce919 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:29.433106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209fd654-1497-4c81-ad32-d120b1e69be8 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:29.469768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a779f62-68ab-4458-8761-ac1acaaa3ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:29.470671 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a779f62-68ab-4458-8761-ac1acaaa3ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:29.470956 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a779f62-68ab-4458-8761-ac1acaaa3ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:29.471236 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a779f62-68ab-4458-8761-ac1acaaa3ff1 None admin] PATH_INFO: `/v3/groups/0dc59d8cece14689b7d92b31c02b0986` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:29.472454 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a779f62-68ab-4458-8761-ac1acaaa3ff1 None admin] RBAC: Authorizing `identity:get_group(group_id=0dc59d8cece14689b7d92b31c02b0986)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:29.474046 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a779f62-68ab-4458-8761-ac1acaaa3ff1 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:29.475767 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 222/440] 10.222.0.13 () {56 vars in 1180 bytes} [Fri May 27 19:30:29 2022] GET /identity/v3/groups/0dc59d8cece14689b7d92b31c02b0986 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:29.482795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a779f62-68ab-4458-8761-ac1acaaa3ff1 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:29.532092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:29.536834 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3658054-cdfe-445c-957c-64a7740ce919 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:29.573692 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c6fde2-b0d6-48aa-a1bb-bc0d88b911cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:29.574575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6fde2-b0d6-48aa-a1bb-bc0d88b911cf None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:29.574957 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6fde2-b0d6-48aa-a1bb-bc0d88b911cf None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:29.575291 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c6fde2-b0d6-48aa-a1bb-bc0d88b911cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:29.584224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:29.590457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c6fde2-b0d6-48aa-a1bb-bc0d88b911cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:29.598720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-b6c6fde2-b0d6-48aa-a1bb-bc0d88b911cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:30:29.598720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:29.598720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:29.598720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:29.598720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:29.598720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:29.598720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:29.598720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:29.598720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:29.598720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:29.598720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:29.598720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:29.598720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:29.598720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:29.598720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:29.598720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:29.606249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:29.606249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:29.606249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:29.606249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:29.606249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:29.606249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:29.606249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:29.606249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:29.606249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:29.606249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:29.606249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:29.606249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:29.606249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:29.606249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:29.606249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:29.606249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:29.607045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:29.607045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:29.607045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:29.607045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:29.607045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:29.607045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:29.607045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:29.607045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:29.607045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:29.607045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:29.607045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:30:29.607045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:29.607045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 219/441] 10.222.0.13 () {56 vars in 1114 bytes} [Fri May 27 19:30:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:29.607045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1db627-731e-49dc-b4a7-d7c866b6c6ff None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:29.607764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1db627-731e-49dc-b4a7-d7c866b6c6ff None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:29.607764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1db627-731e-49dc-b4a7-d7c866b6c6ff None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:29.607764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 220/442] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:29.607764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1db627-731e-49dc-b4a7-d7c866b6c6ff None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:29.613567 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a779f62-68ab-4458-8761-ac1acaaa3ff1 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:29.648657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692a88b-a852-47ea-a8dc-186413493edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:29.649919 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692a88b-a852-47ea-a8dc-186413493edf None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:29.650211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692a88b-a852-47ea-a8dc-186413493edf None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:29.650483 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692a88b-a852-47ea-a8dc-186413493edf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:29.651329 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692a88b-a852-47ea-a8dc-186413493edf None admin] RBAC: Adding query filter params (group.id=0dc59d8cece14689b7d92b31c02b0986, role.id=cb43a575e47147c799eec2f362f4fe0e, scope.project.id=7abea439f4624f6680796de4a97fc13b) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:29.651693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692a88b-a852-47ea-a8dc-186413493edf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:29.653158 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692a88b-a852-47ea-a8dc-186413493edf None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:29.659518 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 223/443] 10.222.0.13 () {56 vars in 1376 bytes} [Fri May 27 19:30:29 2022] GET /identity/v3/role_assignments?group.id=0dc59d8cece14689b7d92b31c02b0986&scope.project.id=7abea439f4624f6680796de4a97fc13b&role.id=cb43a575e47147c799eec2f362f4fe0e => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:29.662387 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abba31-601d-4950-b10a-847541893d0f None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:29.663004 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abba31-601d-4950-b10a-847541893d0f None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:29.663098 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abba31-601d-4950-b10a-847541893d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:29.683791 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-90abba31-601d-4950-b10a-847541893d0f None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:29.685045 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-90abba31-601d-4950-b10a-847541893d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:29.733188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1db627-731e-49dc-b4a7-d7c866b6c6ff None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:29.776085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 224/444] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:29.776391 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793faeb-bace-41bc-b260-a759ee60d93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:29.777133 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793faeb-bace-41bc-b260-a759ee60d93e None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:29.777417 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793faeb-bace-41bc-b260-a759ee60d93e None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:29.777696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793faeb-bace-41bc-b260-a759ee60d93e None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:29.778586 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793faeb-bace-41bc-b260-a759ee60d93e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:29.778893 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793faeb-bace-41bc-b260-a759ee60d93e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:29.780308 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793faeb-bace-41bc-b260-a759ee60d93e None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:29.791420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 221/445] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:30:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:29.796992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90abba31-601d-4950-b10a-847541893d0f None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:29.800045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793faeb-bace-41bc-b260-a759ee60d93e None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:29.829459 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:29.890261 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:29.929424 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793faeb-bace-41bc-b260-a759ee60d93e None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:29.947477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90abba31-601d-4950-b10a-847541893d0f None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:29.977224 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbaf813-fb05-4b3c-ba37-58e11917c781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:29.978217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbaf813-fb05-4b3c-ba37-58e11917c781 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:29.978522 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbaf813-fb05-4b3c-ba37-58e11917c781 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:29.978805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbaf813-fb05-4b3c-ba37-58e11917c781 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:29.995326 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541cc59c-3ec6-4561-8c7b-2a1f9cc8e88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:29.996001 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541cc59c-3ec6-4561-8c7b-2a1f9cc8e88a None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:29.996367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541cc59c-3ec6-4561-8c7b-2a1f9cc8e88a None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:29.996650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541cc59c-3ec6-4561-8c7b-2a1f9cc8e88a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:29.998126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbaf813-fb05-4b3c-ba37-58e11917c781 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:30.007053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-cfbaf813-fb05-4b3c-ba37-58e11917c781 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:30:30.007053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:30.007053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:30.007053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:30.007053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:30.007053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:30.007053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:30.007053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:30.007053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:30.007053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:30.007053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:30.007053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:30.007053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:30.007053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:30.007053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:30.007053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:30.008060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:30.008060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:30.008060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:30.008060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:30.008060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:30.008060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:30.008060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:30.008060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:30.008060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:30.008060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:30.008060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:30.008060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:30.008060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:30.008060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:30.008060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:30.008060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:30.009815 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:30.009815 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:30.009815 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:30.009815 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:30.009815 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:30.009815 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:30.009815 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:30.009815 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:30.009815 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:30.009815 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:30.009815 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:30:30.009815 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:30.009815 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 222/446] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:30:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:30.015618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbaf813-fb05-4b3c-ba37-58e11917c781 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:30.098083 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541cc59c-3ec6-4561-8c7b-2a1f9cc8e88a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:30.099641 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541cc59c-3ec6-4561-8c7b-2a1f9cc8e88a None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:30.123853 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:30.152307 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbaf813-fb05-4b3c-ba37-58e11917c781 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:30.173172 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:30.191849 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef506502-3c03-4386-b483-7c834748a389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:30.192412 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef506502-3c03-4386-b483-7c834748a389 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:30.192691 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef506502-3c03-4386-b483-7c834748a389 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:30.192973 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef506502-3c03-4386-b483-7c834748a389 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:30.193748 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef506502-3c03-4386-b483-7c834748a389 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:30.194049 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef506502-3c03-4386-b483-7c834748a389 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:30.195444 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef506502-3c03-4386-b483-7c834748a389 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:30.205414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 223/447] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:30.211065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 225/448] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:30:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:30:30.212611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef506502-3c03-4386-b483-7c834748a389 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:30.217916 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541cc59c-3ec6-4561-8c7b-2a1f9cc8e88a None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:30.330809 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541cc59c-3ec6-4561-8c7b-2a1f9cc8e88a None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:30.331054 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef506502-3c03-4386-b483-7c834748a389 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:30.366433 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e80ad-0d93-48ee-85ba-81b3c198bbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:30.367308 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e80ad-0d93-48ee-85ba-81b3c198bbea None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:30.368163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e80ad-0d93-48ee-85ba-81b3c198bbea None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:30.368449 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e80ad-0d93-48ee-85ba-81b3c198bbea None admin] PATH_INFO: `/v3/projects/7abea439f4624f6680796de4a97fc13b` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:30.370096 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e80ad-0d93-48ee-85ba-81b3c198bbea None admin] RBAC: Authorizing `identity:get_project(project_id=7abea439f4624f6680796de4a97fc13b)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:30.370727 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece6df6-2720-41f2-9117-6f0174d6b1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:30.371379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece6df6-2720-41f2-9117-6f0174d6b1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:30.371512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e80ad-0d93-48ee-85ba-81b3c198bbea None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:30.371667 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece6df6-2720-41f2-9117-6f0174d6b1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:30.371949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cece6df6-2720-41f2-9117-6f0174d6b1a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:30.373249 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 226/449] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:30:30 2022] GET /identity/v3/projects/7abea439f4624f6680796de4a97fc13b => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:30.378766 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e80ad-0d93-48ee-85ba-81b3c198bbea None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:30.387188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:30.422959 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:30.451646 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece6df6-2720-41f2-9117-6f0174d6b1a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:30.453201 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cece6df6-2720-41f2-9117-6f0174d6b1a6 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:30.494089 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e80ad-0d93-48ee-85ba-81b3c198bbea None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:30.538425 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f1d8c-1e28-49a5-89ff-0e551c4fe318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:30.539293 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f1d8c-1e28-49a5-89ff-0e551c4fe318 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:30.539622 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f1d8c-1e28-49a5-89ff-0e551c4fe318 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:30.539931 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f1d8c-1e28-49a5-89ff-0e551c4fe318 None admin] PATH_INFO: `/v3/groups/0dc59d8cece14689b7d92b31c02b0986` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:30.541806 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f1d8c-1e28-49a5-89ff-0e551c4fe318 None admin] RBAC: Authorizing `identity:get_group(group_id=0dc59d8cece14689b7d92b31c02b0986)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:30.543280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f1d8c-1e28-49a5-89ff-0e551c4fe318 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:30.544939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 227/450] 10.222.0.13 () {56 vars in 1180 bytes} [Fri May 27 19:30:30 2022] GET /identity/v3/groups/0dc59d8cece14689b7d92b31c02b0986 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:30.550842 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f1d8c-1e28-49a5-89ff-0e551c4fe318 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:30.578924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 224/451] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:30:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:30:30.584312 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece6df6-2720-41f2-9117-6f0174d6b1a6 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:30.631277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:30.664893 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f1d8c-1e28-49a5-89ff-0e551c4fe318 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:30.667463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:30.700062 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cece6df6-2720-41f2-9117-6f0174d6b1a6 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:30.705722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3123784-fdeb-41cc-8861-87042de9f5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:30.706171 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3123784-fdeb-41cc-8861-87042de9f5ea None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:30.706453 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3123784-fdeb-41cc-8861-87042de9f5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:30.706731 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3123784-fdeb-41cc-8861-87042de9f5ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:30.707569 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3123784-fdeb-41cc-8861-87042de9f5ea None admin] RBAC: Adding query filter params (group.id=0dc59d8cece14689b7d92b31c02b0986, role.id=eda4ac05bc364224a7f319e7ded54073, scope.project.id=7abea439f4624f6680796de4a97fc13b) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:30.707881 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3123784-fdeb-41cc-8861-87042de9f5ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:30.709385 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3123784-fdeb-41cc-8861-87042de9f5ea None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:30.717272 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 228/452] 10.222.0.13 () {56 vars in 1376 bytes} [Fri May 27 19:30:30 2022] GET /identity/v3/role_assignments?group.id=0dc59d8cece14689b7d92b31c02b0986&scope.project.id=7abea439f4624f6680796de4a97fc13b&role.id=eda4ac05bc364224a7f319e7ded54073 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:30.737303 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ebc5f-c0de-4c53-98fa-04eab4717c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:30.738186 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ebc5f-c0de-4c53-98fa-04eab4717c1c None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:30.738505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ebc5f-c0de-4c53-98fa-04eab4717c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:30.738796 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7ebc5f-c0de-4c53-98fa-04eab4717c1c None admin] PATH_INFO: `/v3/projects/6c2fc958121b4494a2ef337a1cb0b9b4` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:30.740181 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ebc5f-c0de-4c53-98fa-04eab4717c1c None admin] RBAC: Authorizing `identity:get_project(project_id=6c2fc958121b4494a2ef337a1cb0b9b4)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:30.741749 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7ebc5f-c0de-4c53-98fa-04eab4717c1c None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:30.745194 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 225/453] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:30:30 2022] GET /identity/v3/projects/6c2fc958121b4494a2ef337a1cb0b9b4 => generated 344 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:30.751402 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3123784-fdeb-41cc-8861-87042de9f5ea None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:30.872805 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3123784-fdeb-41cc-8861-87042de9f5ea None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:30.912868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec179b0-841b-4895-ae52-b5d45ea9a190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:30.913778 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec179b0-841b-4895-ae52-b5d45ea9a190 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:30.914063 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec179b0-841b-4895-ae52-b5d45ea9a190 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:30.914849 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec179b0-841b-4895-ae52-b5d45ea9a190 None admin] PATH_INFO: `/v3/groups/76ad77868bd74a8185ac8c016f4b9411` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:30.916442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec179b0-841b-4895-ae52-b5d45ea9a190 None admin] RBAC: Authorizing `identity:get_group(group_id=76ad77868bd74a8185ac8c016f4b9411)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:30.917956 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec179b0-841b-4895-ae52-b5d45ea9a190 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:30.919603 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 229/454] 10.222.0.13 () {56 vars in 1180 bytes} [Fri May 27 19:30:30 2022] GET /identity/v3/groups/76ad77868bd74a8185ac8c016f4b9411 => generated 229 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:30.925619 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ebc5f-c0de-4c53-98fa-04eab4717c1c None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:31.037622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7ebc5f-c0de-4c53-98fa-04eab4717c1c None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:31.050705 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:31.075125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695cbea-314b-449e-9299-6de5cd0e0b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:31.075769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695cbea-314b-449e-9299-6de5cd0e0b43 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:31.076052 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695cbea-314b-449e-9299-6de5cd0e0b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:31.076323 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695cbea-314b-449e-9299-6de5cd0e0b43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:31.077182 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695cbea-314b-449e-9299-6de5cd0e0b43 None admin] RBAC: Adding query filter params (group.id=76ad77868bd74a8185ac8c016f4b9411, role.id=6505bbb129404436966ee59aa7a01d8e, scope.project.id=6c2fc958121b4494a2ef337a1cb0b9b4) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:31.077492 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695cbea-314b-449e-9299-6de5cd0e0b43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:31.078923 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695cbea-314b-449e-9299-6de5cd0e0b43 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:31.084977 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 226/455] 10.222.0.13 () {56 vars in 1376 bytes} [Fri May 27 19:30:30 2022] GET /identity/v3/role_assignments?group.id=76ad77868bd74a8185ac8c016f4b9411&scope.project.id=6c2fc958121b4494a2ef337a1cb0b9b4&role.id=6505bbb129404436966ee59aa7a01d8e => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:34.404300 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256fbf23-8097-4df3-954c-f8e99d58c891 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:34.405219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256fbf23-8097-4df3-954c-f8e99d58c891 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:34.405378 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256fbf23-8097-4df3-954c-f8e99d58c891 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:34.408299 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 230/456] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:34.415097 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27397c0-b577-444e-b420-980d155c9c54 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:34.416040 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27397c0-b577-444e-b420-980d155c9c54 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:34.416040 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27397c0-b577-444e-b420-980d155c9c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:34.452385 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-f27397c0-b577-444e-b420-980d155c9c54 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:34.458645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-f27397c0-b577-444e-b420-980d155c9c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:34.637957 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 227/457] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 19:30:34.687347 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256fbf23-8097-4df3-954c-f8e99d58c891 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:34.767563 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff35f7-2776-43f0-930b-c9f3a2feae27 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:34.768302 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff35f7-2776-43f0-930b-c9f3a2feae27 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:34.768799 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ff35f7-2776-43f0-930b-c9f3a2feae27 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:34.771002 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 228/458] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:34.778909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29743d1-5a14-4e9a-962b-4f503ee91495 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:34.779594 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29743d1-5a14-4e9a-962b-4f503ee91495 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:34.780236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29743d1-5a14-4e9a-962b-4f503ee91495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:34.785127 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:34.809652 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-c29743d1-5a14-4e9a-962b-4f503ee91495 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:34.813033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-c29743d1-5a14-4e9a-962b-4f503ee91495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:34.925551 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 229/459] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 19:30:34.928997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256fbf23-8097-4df3-954c-f8e99d58c891 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:34.994618 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855de34-7e7a-4476-bc1e-6fdf330a1cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:34.997766 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855de34-7e7a-4476-bc1e-6fdf330a1cff None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:34.998103 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855de34-7e7a-4476-bc1e-6fdf330a1cff None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:34.998427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855de34-7e7a-4476-bc1e-6fdf330a1cff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:34.999077 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad35b04-3351-45f5-b466-33c1698d9471 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:34.999721 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad35b04-3351-45f5-b466-33c1698d9471 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:35.000372 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad35b04-3351-45f5-b466-33c1698d9471 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:35.003311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 230/460] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:35.004887 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad35b04-3351-45f5-b466-33c1698d9471 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:35.019327 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855de34-7e7a-4476-bc1e-6fdf330a1cff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:35.021779 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855de34-7e7a-4476-bc1e-6fdf330a1cff None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:35.042896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-1855de34-7e7a-4476-bc1e-6fdf330a1cff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:35.042896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:35.042896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:35.042896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:35.042896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:35.042896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:35.042896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:35.042896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:35.042896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:35.042896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:35.042896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:35.042896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:35.042896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:35.042896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:35.042896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:35.044634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:35.044634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:35.044634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:35.044634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:35.044634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:35.044634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:35.044634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:35.044634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:35.044634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:35.044634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:35.044634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:35.044634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:35.044634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:35.044634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:35.044634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:35.044634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:35.044634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:35.047078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:35.047078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:35.047078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:35.047078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:35.047078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 231/461] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:30:35.052770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21248e7-484b-42e2-966d-bfb9e00ac493 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:35.053087 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21248e7-484b-42e2-966d-bfb9e00ac493 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:35.053623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21248e7-484b-42e2-966d-bfb9e00ac493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:35.058836 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:35.077195 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-e21248e7-484b-42e2-966d-bfb9e00ac493 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:35.078407 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-e21248e7-484b-42e2-966d-bfb9e00ac493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:35.143398 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad35b04-3351-45f5-b466-33c1698d9471 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:35.168213 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 232/462] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:35.170422 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e07435-5ffa-499a-9af2-bf6bd92275ce None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:35.170889 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e07435-5ffa-499a-9af2-bf6bd92275ce None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:35.171179 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e07435-5ffa-499a-9af2-bf6bd92275ce None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:35.172646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 233/463] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:35.178035 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e07435-5ffa-499a-9af2-bf6bd92275ce None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:35.191040 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddcc69-2968-4b57-bddf-e45eab05caab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:35.192291 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddcc69-2968-4b57-bddf-e45eab05caab None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:35.192857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddcc69-2968-4b57-bddf-e45eab05caab None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:35.193412 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddcc69-2968-4b57-bddf-e45eab05caab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:35.204150 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:35.207019 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddcc69-2968-4b57-bddf-e45eab05caab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:35.209244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddcc69-2968-4b57-bddf-e45eab05caab None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:35.235784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-04ddcc69-2968-4b57-bddf-e45eab05caab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:35.235784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:35.235784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:35.235784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:35.235784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:35.235784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:35.235784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:35.235784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:35.235784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:35.235784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:35.235784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:35.235784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:35.235784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:35.235784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:35.235784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:35.239268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:35.239268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:35.239268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:35.239268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:35.239268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:35.239268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:35.239268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:35.239268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:35.239268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:35.239268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:35.239268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:35.239268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:35.239268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:35.239268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:35.239268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:35.239268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:35.239268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:35.242875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:35.242875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:35.242875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:35.242875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:35.242875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 231/464] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:35.242875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c3f14-e9d6-400f-a70d-35b870449caf None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:35.242875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c3f14-e9d6-400f-a70d-35b870449caf None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:35.242875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245c3f14-e9d6-400f-a70d-35b870449caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:35.260822 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-245c3f14-e9d6-400f-a70d-35b870449caf None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:35.261977 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-245c3f14-e9d6-400f-a70d-35b870449caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:35.321435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e07435-5ffa-499a-9af2-bf6bd92275ce None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:35.353013 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 232/465] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:35.353013 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c3f14-e9d6-400f-a70d-35b870449caf None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:35.372631 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb0660-e513-4c1e-aacf-51df59616b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:35.374484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb0660-e513-4c1e-aacf-51df59616b6d None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:35.375020 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb0660-e513-4c1e-aacf-51df59616b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:35.375675 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb0660-e513-4c1e-aacf-51df59616b6d None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:35.376892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb0660-e513-4c1e-aacf-51df59616b6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:35.377255 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb0660-e513-4c1e-aacf-51df59616b6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:35.378773 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb0660-e513-4c1e-aacf-51df59616b6d None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:35.388201 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:35.395139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 234/466] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:35.396068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb0660-e513-4c1e-aacf-51df59616b6d None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:35.512786 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245c3f14-e9d6-400f-a70d-35b870449caf None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:35.515754 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:35.546517 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb0660-e513-4c1e-aacf-51df59616b6d None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:35.571126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e5ac6-f916-4fdb-8ffc-0d0a0e7c659a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:35.572329 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e5ac6-f916-4fdb-8ffc-0d0a0e7c659a None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:35.572624 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e5ac6-f916-4fdb-8ffc-0d0a0e7c659a None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:35.572905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e5ac6-f916-4fdb-8ffc-0d0a0e7c659a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:35.586950 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e5ac6-f916-4fdb-8ffc-0d0a0e7c659a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:35.589779 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e5ac6-f916-4fdb-8ffc-0d0a0e7c659a None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:35.594538 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b7293-26bc-4407-b217-7b6e8528edc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:35.595654 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b7293-26bc-4407-b217-7b6e8528edc0 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:35.596191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b7293-26bc-4407-b217-7b6e8528edc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:35.596652 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6b7293-26bc-4407-b217-7b6e8528edc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:35.598040 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b7293-26bc-4407-b217-7b6e8528edc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:35.598583 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b7293-26bc-4407-b217-7b6e8528edc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:35.601145 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6b7293-26bc-4407-b217-7b6e8528edc0 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:35.610292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-c05e5ac6-f916-4fdb-8ffc-0d0a0e7c659a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:35.610292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:35.610292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:35.610292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:35.610292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:35.610292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:35.610292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:35.610292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:35.610292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:35.610292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:35.610292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:35.610292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:35.610292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:35.610292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:35.610292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:35.611253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:35.611253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:35.611253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:35.611253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:35.611253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:35.611253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:35.611253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:35.611253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:35.611253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:35.611253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:35.611253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:35.611253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:35.611253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:35.611253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:35.611253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:35.611253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:35.611253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:35.612128 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:35.612128 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:35.612128 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:35.612128 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:35.612128 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 233/467] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:35.613255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e5ac6-f916-4fdb-8ffc-0d0a0e7c659a None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:35.625370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 235/468] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:35.628286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b7293-26bc-4407-b217-7b6e8528edc0 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:35.706415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:35.749986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e5ac6-f916-4fdb-8ffc-0d0a0e7c659a None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:35.763118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6b7293-26bc-4407-b217-7b6e8528edc0 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:35.793809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6aff9-9390-475c-8d22-c2784b4ca3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:35.795151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6aff9-9390-475c-8d22-c2784b4ca3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:35.795681 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6aff9-9390-475c-8d22-c2784b4ca3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:35.796166 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6aff9-9390-475c-8d22-c2784b4ca3e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:35.803323 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86177125-13fa-4b31-9258-5c998119e245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:35.804042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86177125-13fa-4b31-9258-5c998119e245 None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:35.804350 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86177125-13fa-4b31-9258-5c998119e245 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:35.804635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86177125-13fa-4b31-9258-5c998119e245 None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:35.805415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86177125-13fa-4b31-9258-5c998119e245 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:35.806962 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86177125-13fa-4b31-9258-5c998119e245 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:35.809883 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6aff9-9390-475c-8d22-c2784b4ca3e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:35.811564 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6aff9-9390-475c-8d22-c2784b4ca3e2 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:35.816043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:35.833521 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-88a6aff9-9390-475c-8d22-c2784b4ca3e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:35.833521 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:35.833521 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:35.833521 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:35.833521 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:35.833521 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:35.833521 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:35.833521 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:35.833521 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:35.833521 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:35.833521 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:35.833521 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:35.833521 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:35.833521 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:35.833521 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:35.834611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:35.834611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:35.834611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:35.834611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:35.834611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:35.834611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:35.834611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:35.834611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:35.834611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:35.834611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:35.834611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:35.834611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:35.834611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:35.834611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:35.834611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:35.834611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:35.834611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:35.835519 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:35.835519 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:35.835519 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:35.835519 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:35.835519 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 234/469] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:35.835519 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6aff9-9390-475c-8d22-c2784b4ca3e2 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:35.940980 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 236/470] 10.222.0.13 () {60 vars in 1132 bytes} [Fri May 27 19:30:35 2022] POST /identity/v3/users => generated 246 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:30:35.940980 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86177125-13fa-4b31-9258-5c998119e245 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:35.981244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6aff9-9390-475c-8d22-c2784b4ca3e2 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:36.011739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:36.031436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390190ba-9712-4830-b96b-6cd5d0212c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:36.032624 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390190ba-9712-4830-b96b-6cd5d0212c43 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:36.032998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390190ba-9712-4830-b96b-6cd5d0212c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:36.033374 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390190ba-9712-4830-b96b-6cd5d0212c43 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:36.034138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390190ba-9712-4830-b96b-6cd5d0212c43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:36.034450 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390190ba-9712-4830-b96b-6cd5d0212c43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:36.036058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390190ba-9712-4830-b96b-6cd5d0212c43 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:36.054681 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 235/471] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:36.054681 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390190ba-9712-4830-b96b-6cd5d0212c43 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:36.085295 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86177125-13fa-4b31-9258-5c998119e245 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:36.139081 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccdc115-d269-4b9e-8bf7-192048bc7b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:36.140096 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdc115-d269-4b9e-8bf7-192048bc7b9c None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:36.140368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdc115-d269-4b9e-8bf7-192048bc7b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:36.140647 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccdc115-d269-4b9e-8bf7-192048bc7b9c None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:36.141413 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccdc115-d269-4b9e-8bf7-192048bc7b9c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:36.142906 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccdc115-d269-4b9e-8bf7-192048bc7b9c None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:36.191757 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390190ba-9712-4830-b96b-6cd5d0212c43 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:36.232266 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e77ef-a982-47b5-9d1e-d36c4ba0547f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:36.232942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e77ef-a982-47b5-9d1e-d36c4ba0547f None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:36.233277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e77ef-a982-47b5-9d1e-d36c4ba0547f None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:36.233552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e77ef-a982-47b5-9d1e-d36c4ba0547f None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:36.234282 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e77ef-a982-47b5-9d1e-d36c4ba0547f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:36.234586 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e77ef-a982-47b5-9d1e-d36c4ba0547f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:36.238787 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e77ef-a982-47b5-9d1e-d36c4ba0547f None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:36.245199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 237/472] 10.222.0.13 () {60 vars in 1132 bytes} [Fri May 27 19:30:35 2022] POST /identity/v3/users => generated 248 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:30:36.245199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97298d8-e488-4f10-8d19-795988f65e2a None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:36.245199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97298d8-e488-4f10-8d19-795988f65e2a None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:36.245199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97298d8-e488-4f10-8d19-795988f65e2a None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:36.245199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 238/473] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:36.247733 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97298d8-e488-4f10-8d19-795988f65e2a None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:36.260704 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 236/474] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:36.262220 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a16e3-dcef-458c-961a-a091392fd66f None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:36.262509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a16e3-dcef-458c-961a-a091392fd66f None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:36.262784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a16e3-dcef-458c-961a-a091392fd66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:36.283362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-1a8a16e3-dcef-458c-961a-a091392fd66f None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:36.284618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-1a8a16e3-dcef-458c-961a-a091392fd66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:36.333681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:36.352749 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:36.382121 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 237/475] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:36.384840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a16e3-dcef-458c-961a-a091392fd66f None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:36.439768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97298d8-e488-4f10-8d19-795988f65e2a None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:36.503684 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28c63a-99e1-4e37-a59a-96c19cb932b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:36.504631 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28c63a-99e1-4e37-a59a-96c19cb932b8 None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:36.504903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28c63a-99e1-4e37-a59a-96c19cb932b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:36.505206 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28c63a-99e1-4e37-a59a-96c19cb932b8 None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:36.508470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f28c63a-99e1-4e37-a59a-96c19cb932b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:36.508470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f28c63a-99e1-4e37-a59a-96c19cb932b8 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:36.557072 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a16e3-dcef-458c-961a-a091392fd66f None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:36.607155 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da79ee-7e5a-4801-b536-b98837103ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:36.607155 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da79ee-7e5a-4801-b536-b98837103ac9 None admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:36.613032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da79ee-7e5a-4801-b536-b98837103ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:36.613032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da79ee-7e5a-4801-b536-b98837103ac9 None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:36.613032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17da79ee-7e5a-4801-b536-b98837103ac9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:36.613032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17da79ee-7e5a-4801-b536-b98837103ac9 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:36.642037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 239/476] 10.222.0.13 () {60 vars in 1132 bytes} [Fri May 27 19:30:36 2022] POST /identity/v3/users => generated 249 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:30:36.646037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28c63a-99e1-4e37-a59a-96c19cb932b8 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:36.764822 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 238/477] 10.222.0.13 () {60 vars in 1132 bytes} [Fri May 27 19:30:36 2022] POST /identity/v3/users => generated 248 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:30:36.868057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28c63a-99e1-4e37-a59a-96c19cb932b8 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:36.885035 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:36.913661 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f380a89-59ac-4c93-9b5f-26722de10c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:36.914383 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f380a89-59ac-4c93-9b5f-26722de10c8e None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:36.914668 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f380a89-59ac-4c93-9b5f-26722de10c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:36.914979 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f380a89-59ac-4c93-9b5f-26722de10c8e None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:36.916279 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f380a89-59ac-4c93-9b5f-26722de10c8e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:36.917919 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f380a89-59ac-4c93-9b5f-26722de10c8e None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:36.946236 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.sql.core [None req-7f380a89-59ac-4c93-9b5f-26722de10c8e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 19:30:36.946236 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 27 19:30:36.946236 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [parameters: {'id': 'd8a1a7792f504fc0bcd11c08497a697f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 27 19:30:36.946236 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80635) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 19:30:36.972536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-7f380a89-59ac-4c93-9b5f-26722de10c8e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 27 19:30:36.972536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:36.972536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 19:30:36.972536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 19:30:36.972536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 19:30:36.972536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 19:30:36.972536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 19:30:36.972536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application result = self._query(query) May 27 19:30:36.972536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 19:30:36.972536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application conn.query(q) May 27 19:30:36.972536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 19:30:36.972536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 19:30:36.972536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 19:30:36.972536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application result.read() May 27 19:30:36.972536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 19:30:36.974341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 19:30:36.974341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 19:30:36.974341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 19:30:36.974341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 19:30:36.974341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 19:30:36.974341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 19:30:36.974341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 19:30:36.974341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 19:30:36.974341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:36.974341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 19:30:36.974341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:36.974341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:36.974341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 19:30:36.974341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 19:30:36.974341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 27 19:30:36.974341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return ref.to_dict() May 27 19:30:36.974341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 19:30:36.974341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application next(self.gen) May 27 19:30:36.976057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 19:30:36.976057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application yield resource May 27 19:30:36.976057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 19:30:36.976057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application next(self.gen) May 27 19:30:36.976057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 19:30:36.976057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self.session.rollback() May 27 19:30:36.976057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 19:30:36.976057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self.force_reraise() May 27 19:30:36.976057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 19:30:36.976057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise self.value May 27 19:30:36.976057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 19:30:36.976057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 19:30:36.976057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 19:30:36.976057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application session.commit() May 27 19:30:36.976057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 19:30:36.976057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 19:30:36.976057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 19:30:36.980186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._prepare_impl() May 27 19:30:36.980186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 19:30:36.980186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self.session.flush() May 27 19:30:36.980186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 19:30:36.980186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._flush(objects) May 27 19:30:36.980186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 19:30:36.980186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 19:30:36.980186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 19:30:36.980186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application compat.raise_( May 27 19:30:36.980186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 19:30:36.980186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception May 27 19:30:36.980186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 19:30:36.980186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application flush_context.execute() May 27 19:30:36.980186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 19:30:36.980186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rec.execute(self) May 27 19:30:36.980186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 19:30:36.980186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 19:30:36.995821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 19:30:36.995821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 19:30:36.995821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 27 19:30:36.995821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application c = connection._execute_20( May 27 19:30:36.995821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 19:30:36.995821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 19:30:36.995821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 19:30:36.995821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 19:30:36.995821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 19:30:36.995821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 19:30:36.995821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 19:30:36.995821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 19:30:36.995821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 19:30:36.995821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 19:30:36.995821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 19:30:36.995821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception May 27 19:30:36.999220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 19:30:36.999220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 19:30:36.999220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 19:30:36.999220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 19:30:36.999220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 19:30:36.999220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application result = self._query(query) May 27 19:30:36.999220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 19:30:36.999220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application conn.query(q) May 27 19:30:36.999220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 19:30:36.999220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 19:30:36.999220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 19:30:36.999220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application result.read() May 27 19:30:36.999220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 19:30:36.999220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 19:30:36.999220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 19:30:36.999220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 19:30:37.002564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 19:30:37.002564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 19:30:37.002564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 19:30:37.002564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 19:30:37.002564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 19:30:37.002564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 27 19:30:37.002564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [parameters: {'id': 'd8a1a7792f504fc0bcd11c08497a697f', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 27 19:30:37.002564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 19:30:37.002564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:37.002564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:30:37.002564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:37.002564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:37.002564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:37.002564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:37.002564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:37.002564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:37.016871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:37.016871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:37.016871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:37.016871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:37.016871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:37.016871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:37.016871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 27 19:30:37.016871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 27 19:30:37.016871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:37.016871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:37.016871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 27 19:30:37.016871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 27 19:30:37.016871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 19:30:37.016871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 19:30:37.016871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 27 19:30:37.016871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:37.020707 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 240/478] 10.222.0.13 () {60 vars in 1131 bytes} [Fri May 27 19:30:36 2022] POST /identity/v3/roles => generated 144 bytes in 335 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:37.020917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da79ee-7e5a-4801-b536-b98837103ac9 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:37.130827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:37.281468 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da79ee-7e5a-4801-b536-b98837103ac9 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:37.345345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c18416-5cbc-4ae5-8833-5a52a18588de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:37.347125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c18416-5cbc-4ae5-8833-5a52a18588de None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:37.347862 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c18416-5cbc-4ae5-8833-5a52a18588de None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:37.348571 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c18416-5cbc-4ae5-8833-5a52a18588de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:37.376306 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c18416-5cbc-4ae5-8833-5a52a18588de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:37.386663 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-36c18416-5cbc-4ae5-8833-5a52a18588de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:30:37.386663 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:37.386663 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:37.386663 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:37.386663 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:37.386663 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:37.386663 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:37.386663 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:37.386663 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:37.386663 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:37.386663 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:37.386663 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:37.386663 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:37.386663 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:37.386663 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:37.386663 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:37.388367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:37.388367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:37.388367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:37.388367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:37.388367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:37.388367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:37.388367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:37.388367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:37.388367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:37.388367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:37.388367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:37.388367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:37.388367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:37.388367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:37.388367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:37.388367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:37.390618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:37.390618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:37.390618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:37.390618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:37.390618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:37.390618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:37.390618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:37.390618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:37.390618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:37.390618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:37.390618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 19:30:37.390618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:37.390618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 239/479] 10.222.0.13 () {56 vars in 1114 bytes} [Fri May 27 19:30:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:37.393625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f380a89-59ac-4c93-9b5f-26722de10c8e None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:37.629954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f380a89-59ac-4c93-9b5f-26722de10c8e None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:37.724299 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4afb67-1be1-4016-bfcf-6c3fbd30342c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:37.724299 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4afb67-1be1-4016-bfcf-6c3fbd30342c None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:37.724299 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4afb67-1be1-4016-bfcf-6c3fbd30342c None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:37.724299 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4afb67-1be1-4016-bfcf-6c3fbd30342c None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:37.725294 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4afb67-1be1-4016-bfcf-6c3fbd30342c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:37.725294 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4afb67-1be1-4016-bfcf-6c3fbd30342c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:37.726956 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4afb67-1be1-4016-bfcf-6c3fbd30342c None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:37.749088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 241/480] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:30:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:39.406088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b33ff8-d0f7-4824-b29a-a853025f6d8c None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:39.406648 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b33ff8-d0f7-4824-b29a-a853025f6d8c None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:39.406648 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b33ff8-d0f7-4824-b29a-a853025f6d8c None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:39.408320 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 240/481] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:39.414785 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e6d7b-a6a6-441e-a80e-c92866531563 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:39.416019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e6d7b-a6a6-441e-a80e-c92866531563 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:39.416211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097e6d7b-a6a6-441e-a80e-c92866531563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:39.438152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-097e6d7b-a6a6-441e-a80e-c92866531563 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:39.440016 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-097e6d7b-a6a6-441e-a80e-c92866531563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:39.463048 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8f60d-5915-4656-83c8-af7e3639c9d8 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:39.463361 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8f60d-5915-4656-83c8-af7e3639c9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:39.463589 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a8f60d-5915-4656-83c8-af7e3639c9d8 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:39.465820 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 241/482] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:39.472449 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056991a0-b6cf-410a-8c10-af87ed59920e None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:39.472735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056991a0-b6cf-410a-8c10-af87ed59920e None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:39.473031 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056991a0-b6cf-410a-8c10-af87ed59920e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:39.516986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-056991a0-b6cf-410a-8c10-af87ed59920e None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:39.517245 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:39.517761 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-056991a0-b6cf-410a-8c10-af87ed59920e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:39.618815 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 242/483] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:39.652821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e6d7b-a6a6-441e-a80e-c92866531563 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:39.657278 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 242/484] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 19:30:39.693326 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056991a0-b6cf-410a-8c10-af87ed59920e None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:39.769362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:39.831021 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097e6d7b-a6a6-441e-a80e-c92866531563 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:39.897343 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d7074-8760-44c9-a750-a014f60daedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:39.898176 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d7074-8760-44c9-a750-a014f60daedd None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:39.898457 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d7074-8760-44c9-a750-a014f60daedd None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:39.898729 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d7074-8760-44c9-a750-a014f60daedd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:39.933287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d7074-8760-44c9-a750-a014f60daedd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:39.941552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-f30d7074-8760-44c9-a750-a014f60daedd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:39.941552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:39.941552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:39.941552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:39.941552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:39.941552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:39.941552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:39.941552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:39.941552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:39.941552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:39.941552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:39.941552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:39.941552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:39.941552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:39.941552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:39.941552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:39.942512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:39.942512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:39.942512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:39.942512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:39.942512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:39.942512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:39.942512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:39.942512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:39.942512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:39.942512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:39.942512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:39.942512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:39.942512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:39.942512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:39.942512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:39.942512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:39.943314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:39.943314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:39.943314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:39.943314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:39.943314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:39.943314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:39.943314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:39.943314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:39.943314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:39.943314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:39.943314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:39.943314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:39.943314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 243/485] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:30:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:30:39.945722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e04105-a2a6-4488-a830-8698206194e3 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:39.945722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e04105-a2a6-4488-a830-8698206194e3 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:39.945722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e04105-a2a6-4488-a830-8698206194e3 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:39.947302 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 244/486] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:39.953285 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e04105-a2a6-4488-a830-8698206194e3 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:39.955750 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056991a0-b6cf-410a-8c10-af87ed59920e None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:39.989327 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:39.998656 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a431cc-40af-4ecb-a5e7-6b51d012c401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:39.999667 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a431cc-40af-4ecb-a5e7-6b51d012c401 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:39.999957 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a431cc-40af-4ecb-a5e7-6b51d012c401 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:40.001112 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a431cc-40af-4ecb-a5e7-6b51d012c401 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:40.014245 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a431cc-40af-4ecb-a5e7-6b51d012c401 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:40.024990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-c1a431cc-40af-4ecb-a5e7-6b51d012c401 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:40.024990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:40.024990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:40.024990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:40.024990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:40.024990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:40.024990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:40.024990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:40.024990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:40.024990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:40.024990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:40.024990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:40.024990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:40.024990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:40.024990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:40.024990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:40.026027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:40.026027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:40.026027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:40.026027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:40.026027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:40.026027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:40.026027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:40.026027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:40.026027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:40.026027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:40.026027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:40.026027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:40.026027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:40.026027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:40.026027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:40.026027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:40.026837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:40.026837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:40.026837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:40.026837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:40.026837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:40.026837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:40.026837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:40.026837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:40.026837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:40.026837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:40.026837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:40.026837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:40.027494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 243/487] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:30:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:40.029353 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0775b-52d4-4db8-b3c9-d7386e8a3174 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:40.029473 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0775b-52d4-4db8-b3c9-d7386e8a3174 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:40.029806 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0775b-52d4-4db8-b3c9-d7386e8a3174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:40.051111 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-80e0775b-52d4-4db8-b3c9-d7386e8a3174 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:40.052379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-80e0775b-52d4-4db8-b3c9-d7386e8a3174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:40.079044 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e04105-a2a6-4488-a830-8698206194e3 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:40.119352 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724f889-d22b-481b-900a-44d8a9ee18f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:40.120197 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724f889-d22b-481b-900a-44d8a9ee18f8 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:40.120326 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724f889-d22b-481b-900a-44d8a9ee18f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:40.120645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724f889-d22b-481b-900a-44d8a9ee18f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:40.121619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e724f889-d22b-481b-900a-44d8a9ee18f8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:40.121785 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e724f889-d22b-481b-900a-44d8a9ee18f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:40.123043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e724f889-d22b-481b-900a-44d8a9ee18f8 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:40.125881 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 244/488] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:40.126785 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0775b-52d4-4db8-b3c9-d7386e8a3174 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:40.135570 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:40.140685 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 245/489] 10.222.0.13 () {56 vars in 1103 bytes} [Fri May 27 19:30:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:40.142519 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc11648-fa24-4872-846d-4d2794aa6102 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:40.143082 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc11648-fa24-4872-846d-4d2794aa6102 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:40.143736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc11648-fa24-4872-846d-4d2794aa6102 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:40.151166 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 246/490] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:40.152467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc11648-fa24-4872-846d-4d2794aa6102 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:40.240898 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0775b-52d4-4db8-b3c9-d7386e8a3174 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:40.268709 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:40.280779 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae14557-ac80-4b25-91dc-61274e0389f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:40.281486 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae14557-ac80-4b25-91dc-61274e0389f0 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:40.281769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae14557-ac80-4b25-91dc-61274e0389f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:40.282048 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae14557-ac80-4b25-91dc-61274e0389f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:40.282799 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae14557-ac80-4b25-91dc-61274e0389f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:40.283105 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae14557-ac80-4b25-91dc-61274e0389f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:40.284511 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae14557-ac80-4b25-91dc-61274e0389f0 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:40.286330 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc11648-fa24-4872-846d-4d2794aa6102 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:40.297870 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 245/491] 10.222.0.13 () {56 vars in 1103 bytes} [Fri May 27 19:30:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:40.297870 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae14557-ac80-4b25-91dc-61274e0389f0 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:40.333177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f7072-df18-433d-9632-561d36d81fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:40.334276 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f7072-df18-433d-9632-561d36d81fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:40.334748 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f7072-df18-433d-9632-561d36d81fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:40.335061 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f7072-df18-433d-9632-561d36d81fb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:40.346692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f7072-df18-433d-9632-561d36d81fb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:40.348173 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f7072-df18-433d-9632-561d36d81fb0 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:40.369373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-c55f7072-df18-433d-9632-561d36d81fb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:40.369373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:40.369373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:40.369373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:40.369373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:40.369373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:40.369373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:40.369373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:40.369373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:40.369373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:40.369373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:40.369373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:40.369373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:40.369373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:40.369373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:40.371290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:40.371290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:40.371290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:40.371290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:40.371290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:40.371290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:40.371290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:40.371290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:40.371290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:40.371290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:40.371290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:40.371290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:40.371290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:40.371290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:40.371290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:40.371290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:40.371290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:40.373206 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:40.373206 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:40.373206 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:40.373206 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:40.376872 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 247/492] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:40.378290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc20185-1de8-47e4-88d2-5b9c6ea33c3e None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:40.378581 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc20185-1de8-47e4-88d2-5b9c6ea33c3e None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:40.378857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc20185-1de8-47e4-88d2-5b9c6ea33c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:40.386230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:40.399834 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-acc20185-1de8-47e4-88d2-5b9c6ea33c3e None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:40.401368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-acc20185-1de8-47e4-88d2-5b9c6ea33c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:40.427647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae14557-ac80-4b25-91dc-61274e0389f0 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:40.473351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a5bf5-119a-4dc4-84d1-f674b2c9931e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:40.474346 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a5bf5-119a-4dc4-84d1-f674b2c9931e None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:40.474652 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a5bf5-119a-4dc4-84d1-f674b2c9931e None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:40.475089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a5bf5-119a-4dc4-84d1-f674b2c9931e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:40.484998 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 248/493] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:40.484998 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc20185-1de8-47e4-88d2-5b9c6ea33c3e None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:40.504016 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04a5bf5-119a-4dc4-84d1-f674b2c9931e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:40.521035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-f04a5bf5-119a-4dc4-84d1-f674b2c9931e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:40.521035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:40.521035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:40.521035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:40.521035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:40.521035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:40.521035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:40.521035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:40.521035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:40.521035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:40.521035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:40.521035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:40.521035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:40.521035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:40.521035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:40.521035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:40.522024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:40.522024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:40.522024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:40.522024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:40.522024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:40.522024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:40.522024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:40.522024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:40.522024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:40.522024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:40.522024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:40.522024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:40.522024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:40.522024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:40.522024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:40.522024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:40.522914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:40.522914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:40.522914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:40.522914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:40.522914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:40.522914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:40.522914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:40.522914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:40.522914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:40.522914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:40.522914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:40.522914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:40.529734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 246/494] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:30:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:40.530516 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a5bf5-119a-4dc4-84d1-f674b2c9931e None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:40.567385 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:40.615226 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc20185-1de8-47e4-88d2-5b9c6ea33c3e None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:40.655811 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a5bf5-119a-4dc4-84d1-f674b2c9931e None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:40.660786 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1148b832-5bb0-4aa2-81a1-8ca4c91a11d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:40.661645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148b832-5bb0-4aa2-81a1-8ca4c91a11d2 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:40.661926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148b832-5bb0-4aa2-81a1-8ca4c91a11d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:40.662206 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148b832-5bb0-4aa2-81a1-8ca4c91a11d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:40.675638 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1148b832-5bb0-4aa2-81a1-8ca4c91a11d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:40.677174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1148b832-5bb0-4aa2-81a1-8ca4c91a11d2 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:40.690255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:40.698102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-1148b832-5bb0-4aa2-81a1-8ca4c91a11d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:40.698102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:40.698102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:40.698102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:40.698102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:40.698102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:40.698102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:40.698102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:40.698102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:40.698102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:40.698102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:40.698102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:40.698102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:40.698102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:40.698102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:40.699669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:40.699669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:40.699669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:40.699669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:40.699669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:40.699669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:40.699669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:40.699669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:40.699669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:40.699669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:40.699669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:40.699669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:40.699669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:40.699669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:40.699669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:40.699669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:40.699669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:40.701027 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:40.701027 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:40.701027 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:40.701027 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:40.701027 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 249/495] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:40.701027 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1148b832-5bb0-4aa2-81a1-8ca4c91a11d2 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:40.704677 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7c3b5-34ac-4f91-89de-59c5ba56bc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:40.705409 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7c3b5-34ac-4f91-89de-59c5ba56bc0a None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:40.706580 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7c3b5-34ac-4f91-89de-59c5ba56bc0a None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:40.706580 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7c3b5-34ac-4f91-89de-59c5ba56bc0a None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:40.706787 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7c3b5-34ac-4f91-89de-59c5ba56bc0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:40.707024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7c3b5-34ac-4f91-89de-59c5ba56bc0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:40.708446 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7c3b5-34ac-4f91-89de-59c5ba56bc0a None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:40.727997 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 247/496] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:40.729969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7c3b5-34ac-4f91-89de-59c5ba56bc0a None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:40.822791 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1148b832-5bb0-4aa2-81a1-8ca4c91a11d2 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:40.848236 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:40.859063 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7c3b5-34ac-4f91-89de-59c5ba56bc0a None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:40.870262 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efaf35-7fa2-4183-ab0b-ccc23642a2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:40.871645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efaf35-7fa2-4183-ab0b-ccc23642a2d1 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:40.871933 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efaf35-7fa2-4183-ab0b-ccc23642a2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:40.872216 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efaf35-7fa2-4183-ab0b-ccc23642a2d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:40.888416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4efaf35-7fa2-4183-ab0b-ccc23642a2d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:40.897721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-d4efaf35-7fa2-4183-ab0b-ccc23642a2d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:40.897721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:40.897721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:40.897721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:40.897721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:40.897721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:40.897721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:40.897721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:40.897721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:40.897721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:40.897721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:40.897721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:40.897721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:40.897721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:40.897721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:40.897721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:40.898727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:40.898727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:40.898727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:40.898727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:40.898727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:40.898727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:40.898727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:40.898727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:40.898727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:40.898727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:40.898727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:40.898727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:40.898727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:40.898727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:40.898727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:40.898727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:40.900368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:40.900368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:40.900368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:40.900368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:40.900368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:40.900368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:40.900368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:40.900368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:40.900368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:40.900368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:40.900368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:40.900368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:40.904221 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 250/497] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:30:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:40.905135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efaf35-7fa2-4183-ab0b-ccc23642a2d1 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:40.907761 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034cd80-5054-4158-bf6c-1d7eb727d743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:40.908640 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034cd80-5054-4158-bf6c-1d7eb727d743 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:40.909128 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034cd80-5054-4158-bf6c-1d7eb727d743 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:40.909416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034cd80-5054-4158-bf6c-1d7eb727d743 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:40.910214 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034cd80-5054-4158-bf6c-1d7eb727d743 None admin] RBAC: Adding query filter params (name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:40.910525 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034cd80-5054-4158-bf6c-1d7eb727d743 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:40.911943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034cd80-5054-4158-bf6c-1d7eb727d743 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:40.926735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 248/498] 10.222.0.13 () {56 vars in 1103 bytes} [Fri May 27 19:30:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:40.926735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034cd80-5054-4158-bf6c-1d7eb727d743 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:40.987212 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:41.034948 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efaf35-7fa2-4183-ab0b-ccc23642a2d1 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:41.039215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034cd80-5054-4158-bf6c-1d7eb727d743 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:41.073800 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb95b4-95bf-4183-8212-b6c20a09e7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:41.074377 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb95b4-95bf-4183-8212-b6c20a09e7e7 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:41.074681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb95b4-95bf-4183-8212-b6c20a09e7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:41.074932 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eb95b4-95bf-4183-8212-b6c20a09e7e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:41.075950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb95b4-95bf-4183-8212-b6c20a09e7e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:41.076173 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb95b4-95bf-4183-8212-b6c20a09e7e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:41.077265 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf6369-5fde-41e4-ae14-e0ac695a1544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:41.077711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eb95b4-95bf-4183-8212-b6c20a09e7e7 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:41.078066 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf6369-5fde-41e4-ae14-e0ac695a1544 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:41.078354 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf6369-5fde-41e4-ae14-e0ac695a1544 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:41.078629 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf6369-5fde-41e4-ae14-e0ac695a1544 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:41.086767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf6369-5fde-41e4-ae14-e0ac695a1544 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:41.088791 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf6369-5fde-41e4-ae14-e0ac695a1544 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:41.094833 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 251/499] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:41.094833 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb95b4-95bf-4183-8212-b6c20a09e7e7 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:41.105805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-efcf6369-5fde-41e4-ae14-e0ac695a1544 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 19:30:41.105805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:41.105805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:41.105805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:41.105805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:41.105805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:41.105805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:41.105805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:41.105805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:41.105805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:41.105805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:41.105805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:41.105805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:30:41.105805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:30:41.105805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:30:41.105805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:30:41.107074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:41.107074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:41.107074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:30:41.107074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:41.107074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:30:41.107074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:41.107074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:41.107074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:41.107074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:41.107074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:41.107074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:41.107074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:41.107074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:41.107074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:41.107074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:41.107074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:41.107074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:41.107074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:41.107999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:41.107999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:41.107999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:30:41.107999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:30:41.107999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:30:41.107999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:30:41.107999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:30:41.107999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:30:41.107999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 19:30:41.107999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:41.107999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 249/500] 10.222.0.13 () {56 vars in 1093 bytes} [Fri May 27 19:30:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:41.107999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf6369-5fde-41e4-ae14-e0ac695a1544 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:41.138042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:41.211767 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eb95b4-95bf-4183-8212-b6c20a09e7e7 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:41.212847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf6369-5fde-41e4-ae14-e0ac695a1544 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:41.224089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:41.248133 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb8ceb-0c9a-481b-95ed-ffa4c8b37bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:41.248979 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb8ceb-0c9a-481b-95ed-ffa4c8b37bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:41.249277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb8ceb-0c9a-481b-95ed-ffa4c8b37bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:41.249560 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb8ceb-0c9a-481b-95ed-ffa4c8b37bd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:41.250446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb8ceb-0c9a-481b-95ed-ffa4c8b37bd3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:41.250760 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb8ceb-0c9a-481b-95ed-ffa4c8b37bd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:41.252274 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb8ceb-0c9a-481b-95ed-ffa4c8b37bd3 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:41.261886 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 252/501] 10.222.0.13 () {56 vars in 1103 bytes} [Fri May 27 19:30:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:41.262327 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81801cfd-005f-4ec4-bd19-07f440bb40c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:41.263021 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be07803-cf3d-4c9c-80fe-40f3acafed50 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:41.263266 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81801cfd-005f-4ec4-bd19-07f440bb40c8 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:41.263360 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be07803-cf3d-4c9c-80fe-40f3acafed50 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:41.263557 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81801cfd-005f-4ec4-bd19-07f440bb40c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:41.263650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be07803-cf3d-4c9c-80fe-40f3acafed50 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:41.263840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81801cfd-005f-4ec4-bd19-07f440bb40c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:41.268901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 253/502] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:41.268901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be07803-cf3d-4c9c-80fe-40f3acafed50 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:41.276848 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81801cfd-005f-4ec4-bd19-07f440bb40c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:41.278380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81801cfd-005f-4ec4-bd19-07f440bb40c8 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:41.296767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-81801cfd-005f-4ec4-bd19-07f440bb40c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:41.296767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:41.296767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:41.296767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:41.296767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:41.296767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:41.296767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:41.296767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:41.296767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:41.296767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:41.296767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:41.296767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:41.296767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:41.296767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:41.296767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:41.298244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:41.298244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:41.298244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:41.298244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:41.298244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:41.298244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:41.298244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:41.298244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:41.298244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:41.298244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:41.298244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:41.298244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:41.298244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:41.298244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:41.298244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:41.298244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:41.298244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:41.299101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:41.299101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:41.299101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:41.299101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:41.299101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 250/503] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:41.299101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81801cfd-005f-4ec4-bd19-07f440bb40c8 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:41.380522 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:41.385244 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be07803-cf3d-4c9c-80fe-40f3acafed50 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:41.404413 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81801cfd-005f-4ec4-bd19-07f440bb40c8 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:41.422524 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60f6e8-ea4b-4124-8019-8c2b2537d539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:41.423367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60f6e8-ea4b-4124-8019-8c2b2537d539 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:41.423657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60f6e8-ea4b-4124-8019-8c2b2537d539 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:41.423939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60f6e8-ea4b-4124-8019-8c2b2537d539 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:41.431648 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60f6e8-ea4b-4124-8019-8c2b2537d539 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:41.433164 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60f6e8-ea4b-4124-8019-8c2b2537d539 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:41.439820 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ed872e-e0c6-46be-8d5a-597960d1bfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:41.440384 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed872e-e0c6-46be-8d5a-597960d1bfbc None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:41.440555 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed872e-e0c6-46be-8d5a-597960d1bfbc None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:41.440806 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ed872e-e0c6-46be-8d5a-597960d1bfbc None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:41.441718 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed872e-e0c6-46be-8d5a-597960d1bfbc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:41.442072 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed872e-e0c6-46be-8d5a-597960d1bfbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:41.443478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ed872e-e0c6-46be-8d5a-597960d1bfbc None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:41.449605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-ec60f6e8-ea4b-4124-8019-8c2b2537d539 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 19:30:41.449605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:41.449605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:41.449605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:41.449605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:41.449605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:41.449605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:41.449605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:41.449605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:41.449605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:41.449605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:41.449605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:41.449605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:30:41.449605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:30:41.449605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:30:41.449605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:30:41.451037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:41.451037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:41.451037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:30:41.451037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:41.451037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:30:41.451037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:41.451037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:41.451037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:41.451037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:41.451037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:41.451037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:41.451037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:41.451037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:41.451037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:41.451037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:41.451037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:41.451037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:41.451037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:41.453263 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:41.453263 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:41.453263 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:30:41.453263 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:30:41.453263 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:30:41.453263 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:30:41.453263 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:30:41.453263 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:30:41.453263 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 19:30:41.453263 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:41.453263 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 254/504] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:41.453263 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60f6e8-ea4b-4124-8019-8c2b2537d539 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:41.484448 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 251/505] 10.222.0.13 () {56 vars in 1133 bytes} [Fri May 27 19:30:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:41.485763 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5825d-98f3-43a3-a32a-a84e5e0f0267 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:41.486048 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5825d-98f3-43a3-a32a-a84e5e0f0267 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:41.486318 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5825d-98f3-43a3-a32a-a84e5e0f0267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:41.504031 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-95f5825d-98f3-43a3-a32a-a84e5e0f0267 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:41.505274 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-95f5825d-98f3-43a3-a32a-a84e5e0f0267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:41.524774 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:41.568767 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60f6e8-ea4b-4124-8019-8c2b2537d539 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:41.577643 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 252/506] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:41.578527 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5825d-98f3-43a3-a32a-a84e5e0f0267 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:41.614076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e9f61-ff02-4ac6-a934-fcb123613f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:41.614880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e9f61-ff02-4ac6-a934-fcb123613f1b None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:41.615162 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e9f61-ff02-4ac6-a934-fcb123613f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:41.615441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e9f61-ff02-4ac6-a934-fcb123613f1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:41.627161 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e9f61-ff02-4ac6-a934-fcb123613f1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:41.628656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e9f61-ff02-4ac6-a934-fcb123613f1b None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:41.646982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-1d4e9f61-ff02-4ac6-a934-fcb123613f1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:41.646982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:41.646982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:41.646982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:41.646982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:41.646982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:41.646982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:41.646982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:41.646982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:41.646982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:41.646982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:41.646982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:41.646982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:41.646982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:41.646982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:41.647903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:41.647903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:41.647903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:41.647903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:41.647903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:41.647903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:41.647903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:41.647903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:41.647903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:41.647903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:41.647903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:41.647903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:41.647903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:41.647903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:41.647903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:41.647903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:41.647903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:41.648767 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:41.648767 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:41.648767 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:41.648767 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:41.648767 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 255/507] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:41.649074 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e9f61-ff02-4ac6-a934-fcb123613f1b None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:41.665033 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:41.688536 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5825d-98f3-43a3-a32a-a84e5e0f0267 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:41.725836 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bca9c-3438-47a8-b913-07ffb698eecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:41.726739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bca9c-3438-47a8-b913-07ffb698eecd None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:41.726893 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bca9c-3438-47a8-b913-07ffb698eecd None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:41.727041 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bca9c-3438-47a8-b913-07ffb698eecd None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:41.728059 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250bca9c-3438-47a8-b913-07ffb698eecd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:41.728685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250bca9c-3438-47a8-b913-07ffb698eecd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:41.729800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250bca9c-3438-47a8-b913-07ffb698eecd None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:41.741934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 253/508] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:41.742772 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bca9c-3438-47a8-b913-07ffb698eecd None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:41.756902 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:41.757118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e9f61-ff02-4ac6-a934-fcb123613f1b None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:41.794505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb397518-3bd8-4dd8-bee3-f7d12e5fd780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:41.795189 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb397518-3bd8-4dd8-bee3-f7d12e5fd780 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:41.795475 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb397518-3bd8-4dd8-bee3-f7d12e5fd780 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:41.795759 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb397518-3bd8-4dd8-bee3-f7d12e5fd780 None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:41.796610 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb397518-3bd8-4dd8-bee3-f7d12e5fd780 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:41.796925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb397518-3bd8-4dd8-bee3-f7d12e5fd780 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:41.798422 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb397518-3bd8-4dd8-bee3-f7d12e5fd780 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:41.843484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 256/509] 10.222.0.13 () {56 vars in 1137 bytes} [Fri May 27 19:30:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:41.843484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb397518-3bd8-4dd8-bee3-f7d12e5fd780 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:41.857102 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bca9c-3438-47a8-b913-07ffb698eecd None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:41.894530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678f0fd-5572-4071-a1ff-b98491b36a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:41.895320 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678f0fd-5572-4071-a1ff-b98491b36a88 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:41.895790 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678f0fd-5572-4071-a1ff-b98491b36a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:41.896099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8678f0fd-5572-4071-a1ff-b98491b36a88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:41.918447 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:41.952013 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb397518-3bd8-4dd8-bee3-f7d12e5fd780 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:41.970563 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678f0fd-5572-4071-a1ff-b98491b36a88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:41.972046 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8678f0fd-5572-4071-a1ff-b98491b36a88 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:41.984785 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:41.988615 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041cd78-f92d-49f2-86a2-d1bf9533f958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:41.989450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041cd78-f92d-49f2-86a2-d1bf9533f958 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:41.989752 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041cd78-f92d-49f2-86a2-d1bf9533f958 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:41.990027 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6041cd78-f92d-49f2-86a2-d1bf9533f958 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:42.002243 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6041cd78-f92d-49f2-86a2-d1bf9533f958 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:42.003748 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6041cd78-f92d-49f2-86a2-d1bf9533f958 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:42.021426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-6041cd78-f92d-49f2-86a2-d1bf9533f958 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:42.021426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:42.021426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:42.021426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:42.021426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:42.021426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:42.021426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:42.021426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:42.021426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:42.021426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:42.021426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:42.021426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:42.021426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:42.021426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:42.021426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:42.022520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:42.022520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:42.022520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:42.022520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:42.022520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:42.022520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:42.022520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:42.022520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:42.022520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:42.022520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:42.022520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:42.022520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:42.022520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:42.022520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:42.022520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:42.022520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:42.022520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:42.023400 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:42.023400 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:42.023400 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:42.023400 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:42.023400 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 257/510] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:42.023672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041cd78-f92d-49f2-86a2-d1bf9533f958 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:42.081922 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 254/511] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:30:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:30:42.082789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678f0fd-5572-4071-a1ff-b98491b36a88 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:42.129910 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6041cd78-f92d-49f2-86a2-d1bf9533f958 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:42.153292 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:42.165978 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5064686e-2e64-4b5c-874e-bd0422950ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:42.166660 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064686e-2e64-4b5c-874e-bd0422950ddc None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:42.166947 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064686e-2e64-4b5c-874e-bd0422950ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:42.167229 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5064686e-2e64-4b5c-874e-bd0422950ddc None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:42.168082 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5064686e-2e64-4b5c-874e-bd0422950ddc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:42.168396 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5064686e-2e64-4b5c-874e-bd0422950ddc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:42.169879 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5064686e-2e64-4b5c-874e-bd0422950ddc None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:42.184353 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 258/512] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:42.185516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5064686e-2e64-4b5c-874e-bd0422950ddc None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:42.185878 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8678f0fd-5572-4071-a1ff-b98491b36a88 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:42.198485 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:42.224642 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ebdf5-121c-474b-b7f3-0124968761c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:42.225518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ebdf5-121c-474b-b7f3-0124968761c0 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:42.225799 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ebdf5-121c-474b-b7f3-0124968761c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:42.226077 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ebdf5-121c-474b-b7f3-0124968761c0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:42.234325 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ebdf5-121c-474b-b7f3-0124968761c0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:42.235807 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ebdf5-121c-474b-b7f3-0124968761c0 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:42.253835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-6a5ebdf5-121c-474b-b7f3-0124968761c0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 19:30:42.253835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:42.253835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:42.253835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:42.253835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:42.253835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:42.253835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:42.253835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:42.253835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:42.253835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:42.253835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:42.253835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:42.253835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:30:42.253835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:30:42.253835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:30:42.253835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:30:42.255514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:42.255514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:42.255514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:30:42.255514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:42.255514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:30:42.255514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:42.255514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:42.255514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:42.255514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:42.255514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:42.255514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:42.255514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:42.255514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:42.255514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:42.255514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:42.255514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:42.255514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:42.255514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:42.257005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:42.257005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:42.257005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:30:42.257005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:30:42.257005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:30:42.257005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:30:42.257005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:30:42.257005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:30:42.257005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 19:30:42.257005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:42.257005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 255/513] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:42.257005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ebdf5-121c-474b-b7f3-0124968761c0 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:42.301333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5064686e-2e64-4b5c-874e-bd0422950ddc None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:42.336027 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248b188-91f2-4c79-9d35-96391c42c7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:42.337444 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248b188-91f2-4c79-9d35-96391c42c7de None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:42.337732 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248b188-91f2-4c79-9d35-96391c42c7de None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:42.338008 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248b188-91f2-4c79-9d35-96391c42c7de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:42.361593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ebdf5-121c-474b-b7f3-0124968761c0 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:42.391284 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:42.397820 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73666403-9100-4032-9457-c93a4174b402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:42.398609 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73666403-9100-4032-9457-c93a4174b402 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:42.398895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73666403-9100-4032-9457-c93a4174b402 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:42.399684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73666403-9100-4032-9457-c93a4174b402 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:42.400417 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73666403-9100-4032-9457-c93a4174b402 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:42.400730 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73666403-9100-4032-9457-c93a4174b402 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:42.402259 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73666403-9100-4032-9457-c93a4174b402 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:42.411380 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1248b188-91f2-4c79-9d35-96391c42c7de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:42.412974 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1248b188-91f2-4c79-9d35-96391c42c7de None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:42.420478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 256/514] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:42.420478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73666403-9100-4032-9457-c93a4174b402 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:42.452488 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:42.518807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 259/515] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:30:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:30:42.519697 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248b188-91f2-4c79-9d35-96391c42c7de None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:42.532265 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73666403-9100-4032-9457-c93a4174b402 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:42.568830 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9eaabd-3bb9-4576-9599-9b6b46dc6d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:42.569629 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9eaabd-3bb9-4576-9599-9b6b46dc6d3c None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:42.569908 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9eaabd-3bb9-4576-9599-9b6b46dc6d3c None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:42.570181 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9eaabd-3bb9-4576-9599-9b6b46dc6d3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:42.583368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9eaabd-3bb9-4576-9599-9b6b46dc6d3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:42.585967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9eaabd-3bb9-4576-9599-9b6b46dc6d3c None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:42.607740 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:42.608277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-4b9eaabd-3bb9-4576-9599-9b6b46dc6d3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:42.608277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:42.608277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:42.608277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:42.608277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:42.608277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:42.608277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:42.608277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:42.608277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:42.608277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:42.608277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:42.608277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:42.608277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:42.608277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:42.608277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:42.609364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:42.609364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:42.609364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:42.609364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:42.609364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:42.609364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:42.609364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:42.609364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:42.609364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:42.609364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:42.609364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:42.609364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:42.609364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:42.609364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:42.609364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:42.609364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:42.609364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:42.610717 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:42.610717 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:42.610717 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:42.610717 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:42.613131 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 257/516] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:42.614786 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9eaabd-3bb9-4576-9599-9b6b46dc6d3c None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:42.627417 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248b188-91f2-4c79-9d35-96391c42c7de None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:42.664479 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f253c-07b1-44fc-87d2-a4256c49a730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:42.665418 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f253c-07b1-44fc-87d2-a4256c49a730 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:42.665758 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f253c-07b1-44fc-87d2-a4256c49a730 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:42.666087 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2f253c-07b1-44fc-87d2-a4256c49a730 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:42.673727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2f253c-07b1-44fc-87d2-a4256c49a730 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:42.675300 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2f253c-07b1-44fc-87d2-a4256c49a730 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:42.690639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-3f2f253c-07b1-44fc-87d2-a4256c49a730 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 19:30:42.690639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:42.690639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:42.690639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:42.690639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:42.690639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:42.690639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:42.690639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:42.690639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:42.690639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:42.690639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:42.690639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:42.690639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:30:42.690639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:30:42.690639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:30:42.690639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:30:42.696490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:42.696490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:42.696490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:30:42.696490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:42.696490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:30:42.696490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:42.696490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:42.696490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:42.696490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:42.696490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:42.696490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:42.696490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:42.696490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:42.696490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:42.696490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:42.696490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:42.696490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:42.696490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:42.697592 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:42.697592 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:42.697592 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:30:42.697592 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:30:42.697592 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:30:42.697592 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:30:42.697592 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:30:42.697592 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:30:42.697592 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 19:30:42.697592 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:42.697592 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 260/517] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:30:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:42.697592 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f253c-07b1-44fc-87d2-a4256c49a730 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:42.706188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:42.726244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9eaabd-3bb9-4576-9599-9b6b46dc6d3c None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:42.770185 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f56934-b576-4b59-9ea9-3b74b853d2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:42.771025 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f56934-b576-4b59-9ea9-3b74b853d2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:42.771423 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f56934-b576-4b59-9ea9-3b74b853d2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:42.771862 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f56934-b576-4b59-9ea9-3b74b853d2f9 None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:42.772917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f56934-b576-4b59-9ea9-3b74b853d2f9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:42.773136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f56934-b576-4b59-9ea9-3b74b853d2f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:42.775781 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f56934-b576-4b59-9ea9-3b74b853d2f9 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:42.795050 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 258/518] 10.222.0.13 () {56 vars in 1137 bytes} [Fri May 27 19:30:42 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:42.795921 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f56934-b576-4b59-9ea9-3b74b853d2f9 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:42.830187 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2f253c-07b1-44fc-87d2-a4256c49a730 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:42.871614 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9a5f9-3c89-4886-be85-94680b17cac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:42.872767 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9a5f9-3c89-4886-be85-94680b17cac9 None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:42.872767 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9a5f9-3c89-4886-be85-94680b17cac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:42.873344 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a9a5f9-3c89-4886-be85-94680b17cac9 None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:42.873873 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a9a5f9-3c89-4886-be85-94680b17cac9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:42.875290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a9a5f9-3c89-4886-be85-94680b17cac9 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:42.916334 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f56934-b576-4b59-9ea9-3b74b853d2f9 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:42.950128 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:42.957289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 261/519] 10.222.0.13 () {60 vars in 1132 bytes} [Fri May 27 19:30:42 2022] POST /identity/v3/users => generated 247 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:30:42.958153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9a5f9-3c89-4886-be85-94680b17cac9 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:42.964842 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7263d-42c9-4ab7-9213-4283cbb367b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:42.965739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7263d-42c9-4ab7-9213-4283cbb367b9 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:42.966030 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7263d-42c9-4ab7-9213-4283cbb367b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:42.966285 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e7263d-42c9-4ab7-9213-4283cbb367b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:42.977201 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:42.979903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e7263d-42c9-4ab7-9213-4283cbb367b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:42.981441 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e7263d-42c9-4ab7-9213-4283cbb367b9 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:43.002436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-64e7263d-42c9-4ab7-9213-4283cbb367b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:43.002436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:43.002436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:43.002436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:43.002436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:43.002436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:43.002436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:43.002436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:43.002436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:43.002436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:43.002436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:43.002436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:43.002436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:43.002436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:43.002436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:43.003597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:43.003597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:43.003597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:43.003597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:43.003597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:43.003597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:43.003597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:43.003597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:43.003597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:43.003597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:43.003597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:43.003597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:43.003597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:43.003597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:43.003597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:43.003597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:43.003597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:43.004658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:43.004658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:43.004658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:43.004658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:43.005887 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 259/520] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:43.006842 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7263d-42c9-4ab7-9213-4283cbb367b9 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:43.091637 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a9a5f9-3c89-4886-be85-94680b17cac9 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:43.127280 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e7263d-42c9-4ab7-9213-4283cbb367b9 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:43.130771 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e6b71-cc6d-4b48-8298-0076e3ae4030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:43.131634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e6b71-cc6d-4b48-8298-0076e3ae4030 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:43.132063 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e6b71-cc6d-4b48-8298-0076e3ae4030 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:43.132512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e6b71-cc6d-4b48-8298-0076e3ae4030 None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:43.133986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e6b71-cc6d-4b48-8298-0076e3ae4030 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:43.134527 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e6b71-cc6d-4b48-8298-0076e3ae4030 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:43.136676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e6b71-cc6d-4b48-8298-0076e3ae4030 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:43.152741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 262/521] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:43.153915 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e6b71-cc6d-4b48-8298-0076e3ae4030 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:43.168604 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6545b-a35e-42bb-bae3-d21adb32cbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:43.169329 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6545b-a35e-42bb-bae3-d21adb32cbb9 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:43.169613 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6545b-a35e-42bb-bae3-d21adb32cbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:43.169894 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6545b-a35e-42bb-bae3-d21adb32cbb9 None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:43.170758 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6545b-a35e-42bb-bae3-d21adb32cbb9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:43.171070 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6545b-a35e-42bb-bae3-d21adb32cbb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:43.172581 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6545b-a35e-42bb-bae3-d21adb32cbb9 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:43.230321 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 260/522] 10.222.0.13 () {56 vars in 1139 bytes} [Fri May 27 19:30:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:43.231641 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6545b-a35e-42bb-bae3-d21adb32cbb9 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:43.263795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:43.340982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e6b71-cc6d-4b48-8298-0076e3ae4030 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:43.344263 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:43.399926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196d8a0-4685-4bc5-aead-730627a33683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:43.400794 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196d8a0-4685-4bc5-aead-730627a33683 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:43.401124 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196d8a0-4685-4bc5-aead-730627a33683 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:43.401409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196d8a0-4685-4bc5-aead-730627a33683 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:43.423640 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6545b-a35e-42bb-bae3-d21adb32cbb9 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:43.471605 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9f681-de94-4bec-bea7-1f2b3d98f06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:43.473029 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9f681-de94-4bec-bea7-1f2b3d98f06a None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:43.473489 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9f681-de94-4bec-bea7-1f2b3d98f06a None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:43.473962 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c9f681-de94-4bec-bea7-1f2b3d98f06a None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:43.474867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9f681-de94-4bec-bea7-1f2b3d98f06a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:43.475345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9f681-de94-4bec-bea7-1f2b3d98f06a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:43.476932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c9f681-de94-4bec-bea7-1f2b3d98f06a None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:43.490960 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 261/523] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:43.492181 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9f681-de94-4bec-bea7-1f2b3d98f06a None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:43.520187 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196d8a0-4685-4bc5-aead-730627a33683 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:43.522539 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196d8a0-4685-4bc5-aead-730627a33683 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:43.598105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:43.644190 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c9f681-de94-4bec-bea7-1f2b3d98f06a None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:43.660343 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 263/524] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:30:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:30:43.661143 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196d8a0-4685-4bc5-aead-730627a33683 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:43.667116 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:43.699535 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a044a-3871-41c8-9c35-51fbb00bcfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:43.700347 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a044a-3871-41c8-9c35-51fbb00bcfd5 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:43.700753 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a044a-3871-41c8-9c35-51fbb00bcfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:43.701055 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200a044a-3871-41c8-9c35-51fbb00bcfd5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:43.709063 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200a044a-3871-41c8-9c35-51fbb00bcfd5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:43.710627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200a044a-3871-41c8-9c35-51fbb00bcfd5 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:43.736477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-200a044a-3871-41c8-9c35-51fbb00bcfd5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:30:43.736477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:43.736477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:43.736477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:43.736477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:43.736477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:43.736477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:43.736477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:43.736477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:43.736477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:43.736477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:43.736477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:43.736477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:30:43.736477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:30:43.736477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:30:43.737599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:30:43.737599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:43.737599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:43.737599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:43.737599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:43.737599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:43.737599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:43.737599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:43.737599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:43.737599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:43.737599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:43.737599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:43.737599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:43.737599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:43.737599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:43.737599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:43.737599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:43.739038 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:30:43.739038 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:30:43.739038 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:30:43.739038 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:30:43.739038 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:30:43.739038 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:30:43.739038 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:30:43.739038 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:43.739038 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 262/525] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:30:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:43.739563 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a044a-3871-41c8-9c35-51fbb00bcfd5 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:43.800550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196d8a0-4685-4bc5-aead-730627a33683 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:43.846501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da7d8b-2573-44fb-a15b-d4b0cd809732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:43.847643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da7d8b-2573-44fb-a15b-d4b0cd809732 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:43.848058 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da7d8b-2573-44fb-a15b-d4b0cd809732 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:43.848218 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da7d8b-2573-44fb-a15b-d4b0cd809732 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:43.876174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200a044a-3871-41c8-9c35-51fbb00bcfd5 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:43.893353 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:43.920196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4854b-ec8e-4f8d-9764-fe7f64e1da90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:43.921611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4854b-ec8e-4f8d-9764-fe7f64e1da90 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:43.922110 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4854b-ec8e-4f8d-9764-fe7f64e1da90 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:43.922556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4854b-ec8e-4f8d-9764-fe7f64e1da90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:43.931262 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4854b-ec8e-4f8d-9764-fe7f64e1da90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:43.932745 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4854b-ec8e-4f8d-9764-fe7f64e1da90 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:43.947545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-a3b4854b-ec8e-4f8d-9764-fe7f64e1da90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:30:43.947545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:43.947545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:43.947545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:43.947545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:43.947545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:43.947545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:43.947545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:43.947545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:43.947545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:43.947545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:43.947545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:43.947545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:30:43.947545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:30:43.947545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:30:43.948809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:30:43.948809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:43.948809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:43.948809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:43.948809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:43.948809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:43.948809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:43.948809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:43.948809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:43.948809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:43.948809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:43.948809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:43.948809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:43.948809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:43.948809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:43.948809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:43.948809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:43.950545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:30:43.950545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:30:43.950545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:30:43.950545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:30:43.950545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:30:43.950545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:30:43.950545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:30:43.950545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:43.950545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 263/526] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:30:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:43.950545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4854b-ec8e-4f8d-9764-fe7f64e1da90 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:43.958356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da7d8b-2573-44fb-a15b-d4b0cd809732 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:43.959933 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da7d8b-2573-44fb-a15b-d4b0cd809732 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:43.994918 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:44.088820 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4854b-ec8e-4f8d-9764-fe7f64e1da90 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:44.093365 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 264/527] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:30:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:30:44.094801 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da7d8b-2573-44fb-a15b-d4b0cd809732 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:44.141503 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8160e415-dcdb-4607-aa58-5c953a497a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:44.142322 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160e415-dcdb-4607-aa58-5c953a497a03 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:44.142607 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160e415-dcdb-4607-aa58-5c953a497a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:44.142880 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8160e415-dcdb-4607-aa58-5c953a497a03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:44.155662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8160e415-dcdb-4607-aa58-5c953a497a03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:44.157178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8160e415-dcdb-4607-aa58-5c953a497a03 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:44.175903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-8160e415-dcdb-4607-aa58-5c953a497a03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:44.175903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:44.175903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:44.175903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:44.175903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:44.175903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:44.175903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:44.175903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:44.175903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:44.175903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:44.175903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:44.175903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:44.175903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:44.175903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:44.175903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:44.177200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:44.177200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:44.177200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:44.177200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:44.177200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:44.177200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:44.177200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:44.177200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:44.177200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:44.177200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:44.177200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:44.177200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:44.177200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:44.177200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:44.177200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:44.177200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:44.177200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:44.178423 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:44.178423 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:44.178423 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:44.178423 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:44.178423 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 264/528] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:44.178770 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8160e415-dcdb-4607-aa58-5c953a497a03 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:44.185020 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:44.249091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da7d8b-2573-44fb-a15b-d4b0cd809732 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:44.293049 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e20d4-3084-4b13-afd1-35f1ea99b560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:44.293849 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e20d4-3084-4b13-afd1-35f1ea99b560 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:44.294143 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e20d4-3084-4b13-afd1-35f1ea99b560 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:44.294437 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e20d4-3084-4b13-afd1-35f1ea99b560 None admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:44.295416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e20d4-3084-4b13-afd1-35f1ea99b560 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:44.295857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e20d4-3084-4b13-afd1-35f1ea99b560 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:44.298222 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e20d4-3084-4b13-afd1-35f1ea99b560 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:44.305335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:44.325218 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 265/529] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:30:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:44.326701 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e20d4-3084-4b13-afd1-35f1ea99b560 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:44.334626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8160e415-dcdb-4607-aa58-5c953a497a03 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:44.385826 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2a3c4-bb97-4b5b-9698-29a9d3c8c801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:44.386492 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2a3c4-bb97-4b5b-9698-29a9d3c8c801 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:44.386850 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2a3c4-bb97-4b5b-9698-29a9d3c8c801 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:44.387087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2a3c4-bb97-4b5b-9698-29a9d3c8c801 None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:44.388146 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a2a3c4-bb97-4b5b-9698-29a9d3c8c801 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:44.388464 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a2a3c4-bb97-4b5b-9698-29a9d3c8c801 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:44.390114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a2a3c4-bb97-4b5b-9698-29a9d3c8c801 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:44.410888 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 265/530] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:30:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:44.411801 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2a3c4-bb97-4b5b-9698-29a9d3c8c801 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:44.484136 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e20d4-3084-4b13-afd1-35f1ea99b560 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:44.509924 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:44.538585 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a331f13-0fb5-49a6-94dd-19e6cd65f9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:44.539594 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a331f13-0fb5-49a6-94dd-19e6cd65f9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:44.539944 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a331f13-0fb5-49a6-94dd-19e6cd65f9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:44.540286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a331f13-0fb5-49a6-94dd-19e6cd65f9c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:44.552111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a331f13-0fb5-49a6-94dd-19e6cd65f9c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:44.553789 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a331f13-0fb5-49a6-94dd-19e6cd65f9c1 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:44.579580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-1a331f13-0fb5-49a6-94dd-19e6cd65f9c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:44.579580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:44.579580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:44.579580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:44.579580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:44.579580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:44.579580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:44.579580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:44.579580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:44.579580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:44.579580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:44.579580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:44.579580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:44.579580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:44.579580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:44.581688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:44.581688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:44.581688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:44.581688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:44.581688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:44.581688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:44.581688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:44.581688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:44.581688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:44.581688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:44.581688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:44.581688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:44.581688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:44.581688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:44.581688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:44.581688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:44.581688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:44.583542 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:44.583542 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:44.583542 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:44.583542 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:44.583542 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 266/531] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:44.583542 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a331f13-0fb5-49a6-94dd-19e6cd65f9c1 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:44.594356 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2a3c4-bb97-4b5b-9698-29a9d3c8c801 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:44.656131 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a427a-ef13-48c4-84d8-cdfae0fe3d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:44.656725 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a427a-ef13-48c4-84d8-cdfae0fe3d99 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:44.657047 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a427a-ef13-48c4-84d8-cdfae0fe3d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:44.657329 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a427a-ef13-48c4-84d8-cdfae0fe3d99 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:44.658077 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6a427a-ef13-48c4-84d8-cdfae0fe3d99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:44.658382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6a427a-ef13-48c4-84d8-cdfae0fe3d99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:44.659778 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6a427a-ef13-48c4-84d8-cdfae0fe3d99 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:44.676409 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 266/532] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:44.676409 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a427a-ef13-48c4-84d8-cdfae0fe3d99 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:44.680828 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:44.762992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a331f13-0fb5-49a6-94dd-19e6cd65f9c1 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:44.824070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3950fd-9a6b-431d-b8c3-8fbcb3d1b099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:44.825021 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3950fd-9a6b-431d-b8c3-8fbcb3d1b099 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:44.825493 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3950fd-9a6b-431d-b8c3-8fbcb3d1b099 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:44.825976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3950fd-9a6b-431d-b8c3-8fbcb3d1b099 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:44.827331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3950fd-9a6b-431d-b8c3-8fbcb3d1b099 None admin] RBAC: Adding query filter params (role.id=11e7404599ae49afb5edd22dabe1a5b2, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=7542e1fd160e4b2c9d25accf19aad19f) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:44.827730 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3950fd-9a6b-431d-b8c3-8fbcb3d1b099 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:44.831109 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3950fd-9a6b-431d-b8c3-8fbcb3d1b099 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:44.842794 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 267/533] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:30:44 2022] GET /identity/v3/role_assignments?user.id=7542e1fd160e4b2c9d25accf19aad19f&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=11e7404599ae49afb5edd22dabe1a5b2 => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:44.849190 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a427a-ef13-48c4-84d8-cdfae0fe3d99 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:44.853973 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3950fd-9a6b-431d-b8c3-8fbcb3d1b099 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:44.910841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3af822-787c-4611-9657-14c7221fa812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:44.911164 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3af822-787c-4611-9657-14c7221fa812 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:44.911448 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3af822-787c-4611-9657-14c7221fa812 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:44.911681 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3af822-787c-4611-9657-14c7221fa812 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:44.912509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3af822-787c-4611-9657-14c7221fa812 None admin] RBAC: Adding query filter params (role.id=11e7404599ae49afb5edd22dabe1a5b2, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=1347c0bd4ada4139ad194d423958be6e) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:44.912827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3af822-787c-4611-9657-14c7221fa812 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:44.914950 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3af822-787c-4611-9657-14c7221fa812 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:44.929573 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 267/534] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:30:44 2022] GET /identity/v3/role_assignments?user.id=1347c0bd4ada4139ad194d423958be6e&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=11e7404599ae49afb5edd22dabe1a5b2 => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:44.931205 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3af822-787c-4611-9657-14c7221fa812 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:44.947500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:45.094377 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3950fd-9a6b-431d-b8c3-8fbcb3d1b099 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:45.157325 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:45.178645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4129e1-c3be-466a-a9ac-e67695772d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:45.179673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4129e1-c3be-466a-a9ac-e67695772d42 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:45.180180 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4129e1-c3be-466a-a9ac-e67695772d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:45.180668 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4129e1-c3be-466a-a9ac-e67695772d42 None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:45.181593 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4129e1-c3be-466a-a9ac-e67695772d42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:45.181983 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4129e1-c3be-466a-a9ac-e67695772d42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:45.183977 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4129e1-c3be-466a-a9ac-e67695772d42 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:45.209351 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 268/535] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:45.216247 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4129e1-c3be-466a-a9ac-e67695772d42 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:45.233609 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3af822-787c-4611-9657-14c7221fa812 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:45.296412 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504917b6-de0b-4a9a-97b0-de83030ae92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:45.299351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504917b6-de0b-4a9a-97b0-de83030ae92f None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:45.300076 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504917b6-de0b-4a9a-97b0-de83030ae92f None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:45.301870 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504917b6-de0b-4a9a-97b0-de83030ae92f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:45.323417 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504917b6-de0b-4a9a-97b0-de83030ae92f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:45.329848 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504917b6-de0b-4a9a-97b0-de83030ae92f None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:45.354126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-504917b6-de0b-4a9a-97b0-de83030ae92f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:30:45.354126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:45.354126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:45.354126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:45.354126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:45.354126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:45.354126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:45.354126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:45.354126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:45.354126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:45.354126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:45.354126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:45.354126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:30:45.354126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:30:45.354126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:30:45.355056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:30:45.355056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:45.355056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:45.355056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:45.355056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:45.355056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:45.355056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:45.355056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:45.355056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:45.355056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:45.355056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:45.355056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:45.355056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:45.355056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:45.355056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:45.355056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:45.355056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:45.355917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:30:45.355917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:30:45.355917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:30:45.355917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:30:45.355917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:30:45.355917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:30:45.355917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:30:45.355917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:45.369299 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 268/536] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:30:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 435 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:45.373751 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504917b6-de0b-4a9a-97b0-de83030ae92f None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:45.477935 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:45.492919 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4129e1-c3be-466a-a9ac-e67695772d42 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:45.601725 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2f58a-308a-4e4a-ac60-d5dda0d30ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:45.603131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2f58a-308a-4e4a-ac60-d5dda0d30ecf None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:45.603684 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2f58a-308a-4e4a-ac60-d5dda0d30ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:45.604043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2f58a-308a-4e4a-ac60-d5dda0d30ecf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:45.624849 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2f58a-308a-4e4a-ac60-d5dda0d30ecf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:45.627221 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b2f58a-308a-4e4a-ac60-d5dda0d30ecf None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:45.635397 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504917b6-de0b-4a9a-97b0-de83030ae92f None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:45.668765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-b2b2f58a-308a-4e4a-ac60-d5dda0d30ecf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:30:45.668765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:45.668765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:45.668765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:45.668765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:45.668765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:45.668765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:45.668765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:45.668765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:45.668765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:45.668765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:45.668765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:45.668765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:30:45.668765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:30:45.668765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:30:45.669722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:30:45.669722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:45.669722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:45.669722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:45.669722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:45.669722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:45.669722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:45.669722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:45.669722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:45.669722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:45.669722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:45.669722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:45.669722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:45.669722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:45.669722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:45.669722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:45.669722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:45.670563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:30:45.670563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:30:45.670563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:30:45.670563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:30:45.670563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:30:45.670563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:30:45.670563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:30:45.670563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:45.677543 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 269/537] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:30:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 462 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:45.682334 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2f58a-308a-4e4a-ac60-d5dda0d30ecf None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:45.694680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:45.704328 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd106e5-c0c3-4775-967d-d04d6b103d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:45.705338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd106e5-c0c3-4775-967d-d04d6b103d3d None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:45.705684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd106e5-c0c3-4775-967d-d04d6b103d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:45.706189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd106e5-c0c3-4775-967d-d04d6b103d3d None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:45.707484 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd106e5-c0c3-4775-967d-d04d6b103d3d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:45.707830 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd106e5-c0c3-4775-967d-d04d6b103d3d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:45.709949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd106e5-c0c3-4775-967d-d04d6b103d3d None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:45.728679 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 269/538] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:30:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:45.736523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd106e5-c0c3-4775-967d-d04d6b103d3d None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:45.898144 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd106e5-c0c3-4775-967d-d04d6b103d3d None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:45.905429 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2f58a-308a-4e4a-ac60-d5dda0d30ecf None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:45.956120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271acff-8d33-4b0c-ac08-3e3e448d2841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:45.957174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271acff-8d33-4b0c-ac08-3e3e448d2841 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:45.957707 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271acff-8d33-4b0c-ac08-3e3e448d2841 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:45.958086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271acff-8d33-4b0c-ac08-3e3e448d2841 None admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:45.959304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9271acff-8d33-4b0c-ac08-3e3e448d2841 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:45.959794 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9271acff-8d33-4b0c-ac08-3e3e448d2841 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:45.962691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9271acff-8d33-4b0c-ac08-3e3e448d2841 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:45.968268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05e63e-d08d-4737-81a1-a51fea6cde62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:45.969425 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05e63e-d08d-4737-81a1-a51fea6cde62 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:45.969999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05e63e-d08d-4737-81a1-a51fea6cde62 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:45.970596 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05e63e-d08d-4737-81a1-a51fea6cde62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:45.972099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05e63e-d08d-4737-81a1-a51fea6cde62 None admin] RBAC: Adding query filter params (role.id=11e7404599ae49afb5edd22dabe1a5b2, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=7b7e90ca75d643cbb0e6d6791fc30b45) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:45.972721 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05e63e-d08d-4737-81a1-a51fea6cde62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:45.974985 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05e63e-d08d-4737-81a1-a51fea6cde62 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:45.983977 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 270/539] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:30:45 2022] GET /identity/v3/role_assignments?user.id=7b7e90ca75d643cbb0e6d6791fc30b45&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=11e7404599ae49afb5edd22dabe1a5b2 => generated 259 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:30:45.985970 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14a3fe-a2dc-4f21-9fc0-35781d7e2d4a None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:45.986521 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14a3fe-a2dc-4f21-9fc0-35781d7e2d4a None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:45.987005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa14a3fe-a2dc-4f21-9fc0-35781d7e2d4a None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:45.987765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 270/540] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:30:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:45.989422 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 271/541] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:45.991768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271acff-8d33-4b0c-ac08-3e3e448d2841 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:46.014855 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad85013-75f0-45a6-8c85-ee4e8641a6d0 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:46.015487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad85013-75f0-45a6-8c85-ee4e8641a6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:46.015998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad85013-75f0-45a6-8c85-ee4e8641a6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:46.034380 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:46.048127 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-5ad85013-75f0-45a6-8c85-ee4e8641a6d0 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:46.049507 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-5ad85013-75f0-45a6-8c85-ee4e8641a6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:46.157759 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271acff-8d33-4b0c-ac08-3e3e448d2841 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:46.184046 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:46.246098 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 272/542] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:46.261536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a505e-485c-47ca-908c-86eec1104b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:46.262949 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a505e-485c-47ca-908c-86eec1104b1b None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:46.263599 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a505e-485c-47ca-908c-86eec1104b1b None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:46.264253 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a505e-485c-47ca-908c-86eec1104b1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:46.266038 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a505e-485c-47ca-908c-86eec1104b1b None admin] RBAC: Adding query filter params (role.id=11e7404599ae49afb5edd22dabe1a5b2, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=dd34f15400c4476bbb2161dbe1b8b2b3) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:46.266616 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a505e-485c-47ca-908c-86eec1104b1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:46.268907 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a505e-485c-47ca-908c-86eec1104b1b None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:46.278432 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 271/543] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:30:45 2022] GET /identity/v3/role_assignments?user.id=dd34f15400c4476bbb2161dbe1b8b2b3&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=11e7404599ae49afb5edd22dabe1a5b2 => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:46.308827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad85013-75f0-45a6-8c85-ee4e8641a6d0 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:46.521658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad85013-75f0-45a6-8c85-ee4e8641a6d0 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:46.598207 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b819b5-974b-4e16-a1dd-6e0184f6de18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:46.599205 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b819b5-974b-4e16-a1dd-6e0184f6de18 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:46.599668 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b819b5-974b-4e16-a1dd-6e0184f6de18 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:46.600090 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b819b5-974b-4e16-a1dd-6e0184f6de18 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:46.614089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b819b5-974b-4e16-a1dd-6e0184f6de18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:46.623429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-09b819b5-974b-4e16-a1dd-6e0184f6de18 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:46.623429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:46.623429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:46.623429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:46.623429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:46.623429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:46.623429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:46.623429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:46.623429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:46.623429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:46.623429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:46.623429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:46.623429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:46.623429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:46.623429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:46.623429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:46.624556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:46.624556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:46.624556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:46.624556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:46.624556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:46.624556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:46.624556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:46.624556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:46.624556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:46.624556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:46.624556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:46.624556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:46.624556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:46.624556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:46.624556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:46.624556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:46.625401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:46.625401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:46.625401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:46.625401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:46.625401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:46.625401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:46.625401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:46.625401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:46.625401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:46.625401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:46.625401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:46.625401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:46.629113 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 273/544] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:30:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:46.631739 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a505e-485c-47ca-908c-86eec1104b1b None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:46.780022 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:46.824981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a505e-485c-47ca-908c-86eec1104b1b None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:46.900002 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcabcd2-29e7-459b-97b7-2ddc47a93c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:46.900702 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcabcd2-29e7-459b-97b7-2ddc47a93c9b None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:46.904976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcabcd2-29e7-459b-97b7-2ddc47a93c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:46.904976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcabcd2-29e7-459b-97b7-2ddc47a93c9b None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:46.904976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcabcd2-29e7-459b-97b7-2ddc47a93c9b None admin] RBAC: Adding query filter params (name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:46.904976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcabcd2-29e7-459b-97b7-2ddc47a93c9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:46.904976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcabcd2-29e7-459b-97b7-2ddc47a93c9b None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:46.915265 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 272/545] 10.222.0.13 () {56 vars in 1103 bytes} [Fri May 27 19:30:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:46.922385 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b819b5-974b-4e16-a1dd-6e0184f6de18 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:47.049681 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:47.127568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b819b5-974b-4e16-a1dd-6e0184f6de18 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:47.179159 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc198f-0d39-4803-8703-e07faacd16fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:47.180282 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc198f-0d39-4803-8703-e07faacd16fc None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:47.181422 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc198f-0d39-4803-8703-e07faacd16fc None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:47.181422 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc198f-0d39-4803-8703-e07faacd16fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:47.193713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bc198f-0d39-4803-8703-e07faacd16fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:47.196061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bc198f-0d39-4803-8703-e07faacd16fc None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:47.230379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-31bc198f-0d39-4803-8703-e07faacd16fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:47.230379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:47.230379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:47.230379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:47.230379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:47.230379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:47.230379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:47.230379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:47.230379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:47.230379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:47.230379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:47.230379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:47.230379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:47.230379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:47.230379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:47.232068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:47.232068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:47.232068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:47.232068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:47.232068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:47.232068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:47.232068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:47.232068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:47.232068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:47.232068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:47.232068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:47.232068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:47.232068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:47.232068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:47.232068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:47.232068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:47.232068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:47.235703 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:47.235703 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:47.235703 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:47.235703 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:47.235703 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 274/546] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:47.246258 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcabcd2-29e7-459b-97b7-2ddc47a93c9b None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:47.419798 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d1cec-deb5-4525-bfcf-7605638847fc None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:47.420263 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d1cec-deb5-4525-bfcf-7605638847fc None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:47.420656 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d1cec-deb5-4525-bfcf-7605638847fc None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:47.422553 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcabcd2-29e7-459b-97b7-2ddc47a93c9b None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:47.429803 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 275/547] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:47 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:47.439465 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2045878-a7b9-4cbc-b4a2-61ef631a0905 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:47.439921 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2045878-a7b9-4cbc-b4a2-61ef631a0905 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:47.440412 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2045878-a7b9-4cbc-b4a2-61ef631a0905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:47.471658 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5411a10-4818-4d00-94ed-6432538a6986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:47.472439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5411a10-4818-4d00-94ed-6432538a6986 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:47.472743 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5411a10-4818-4d00-94ed-6432538a6986 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:47.473024 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5411a10-4818-4d00-94ed-6432538a6986 None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:47.473794 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5411a10-4818-4d00-94ed-6432538a6986 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:47.474117 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5411a10-4818-4d00-94ed-6432538a6986 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:47.475714 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5411a10-4818-4d00-94ed-6432538a6986 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:47.486871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:47.491664 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 273/548] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:47.498175 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5411a10-4818-4d00-94ed-6432538a6986 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:47.504476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-b2045878-a7b9-4cbc-b4a2-61ef631a0905 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:47.540893 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-b2045878-a7b9-4cbc-b4a2-61ef631a0905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:47.704337 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5411a10-4818-4d00-94ed-6432538a6986 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:47.708514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 276/549] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:47.748227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2045878-a7b9-4cbc-b4a2-61ef631a0905 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:47.816739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:47.823259 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15654aec-4cb0-4c5f-b84b-26118108ee82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:47.824608 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15654aec-4cb0-4c5f-b84b-26118108ee82 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:47.825269 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15654aec-4cb0-4c5f-b84b-26118108ee82 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:47.825799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15654aec-4cb0-4c5f-b84b-26118108ee82 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:47.847504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15654aec-4cb0-4c5f-b84b-26118108ee82 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:47.849636 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15654aec-4cb0-4c5f-b84b-26118108ee82 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:47.899942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-15654aec-4cb0-4c5f-b84b-26118108ee82 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 19:30:47.899942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:47.899942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:47.899942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:47.899942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:47.899942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:47.899942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:47.899942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:47.899942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:47.899942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:47.899942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:47.899942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:47.899942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:30:47.899942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:30:47.899942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:30:47.899942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:30:47.901056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:47.901056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:47.901056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:30:47.901056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:47.901056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:30:47.901056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:47.901056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:47.901056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:47.901056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:47.901056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:47.901056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:47.901056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:47.901056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:47.901056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:47.901056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:47.901056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:47.901056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:47.901056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:47.902228 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:47.902228 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:47.902228 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:30:47.902228 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:30:47.902228 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:30:47.902228 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:30:47.902228 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:30:47.902228 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:30:47.902228 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 19:30:47.902228 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:47.908536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 274/550] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:30:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:47.909554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15654aec-4cb0-4c5f-b84b-26118108ee82 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:47.961855 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2045878-a7b9-4cbc-b4a2-61ef631a0905 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:48.058886 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e23fb-606e-497a-9652-b2ecf75bf900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:48.059719 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e23fb-606e-497a-9652-b2ecf75bf900 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:48.060006 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e23fb-606e-497a-9652-b2ecf75bf900 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:48.060287 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e23fb-606e-497a-9652-b2ecf75bf900 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:48.067838 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:48.079319 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e23fb-606e-497a-9652-b2ecf75bf900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:48.090754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-1f8e23fb-606e-497a-9652-b2ecf75bf900 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:48.090754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:48.090754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:48.090754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:48.090754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:48.090754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:48.090754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:48.090754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:48.090754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:48.090754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:48.090754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:48.090754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:48.090754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:48.090754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:48.090754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:48.090754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:48.091869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:48.091869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:48.091869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:48.091869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:48.091869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:48.091869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:48.091869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:48.091869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:48.091869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:48.091869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:48.091869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:48.091869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:48.091869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:48.091869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:48.091869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:48.091869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:48.092777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:48.092777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:48.092777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:48.092777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:48.092777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:48.092777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:48.092777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:48.092777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:48.092777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:48.092777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:48.092777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:48.092777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:48.095941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 277/551] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:30:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:30:48.097754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca642f-eb1f-4ae5-8f8d-ac4117154cdb None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:48.098665 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca642f-eb1f-4ae5-8f8d-ac4117154cdb None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:48.099376 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ca642f-eb1f-4ae5-8f8d-ac4117154cdb None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:48.102580 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 278/552] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:48.109557 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca642f-eb1f-4ae5-8f8d-ac4117154cdb None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:48.147427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15654aec-4cb0-4c5f-b84b-26118108ee82 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:48.227812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d68834b-931d-479b-b71e-2a705fed84bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:48.228728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68834b-931d-479b-b71e-2a705fed84bc None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:48.229108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68834b-931d-479b-b71e-2a705fed84bc None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:48.229388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68834b-931d-479b-b71e-2a705fed84bc None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:48.230299 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d68834b-931d-479b-b71e-2a705fed84bc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:48.230619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d68834b-931d-479b-b71e-2a705fed84bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:48.232527 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d68834b-931d-479b-b71e-2a705fed84bc None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:48.282542 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 275/553] 10.222.0.13 () {56 vars in 1135 bytes} [Fri May 27 19:30:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:48.285801 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdea8fe-fe4f-4eec-aad8-f475659fbfe6 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:48.286098 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdea8fe-fe4f-4eec-aad8-f475659fbfe6 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:48.286384 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdea8fe-fe4f-4eec-aad8-f475659fbfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:48.296339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:48.321110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-1bdea8fe-fe4f-4eec-aad8-f475659fbfe6 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:48.324759 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-1bdea8fe-fe4f-4eec-aad8-f475659fbfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:48.338018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ca642f-eb1f-4ae5-8f8d-ac4117154cdb None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:48.451794 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffca204-1466-4019-b8ea-5861196d6189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:48.452534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffca204-1466-4019-b8ea-5861196d6189 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:48.452937 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffca204-1466-4019-b8ea-5861196d6189 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:48.453382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffca204-1466-4019-b8ea-5861196d6189 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:48.454333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffca204-1466-4019-b8ea-5861196d6189 None admin] RBAC: Adding query filter params (name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:48.454740 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffca204-1466-4019-b8ea-5861196d6189 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:48.456251 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffca204-1466-4019-b8ea-5861196d6189 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:48.467235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 279/554] 10.222.0.13 () {56 vars in 1103 bytes} [Fri May 27 19:30:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:48.468115 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffca204-1466-4019-b8ea-5861196d6189 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:48.564004 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 276/555] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:48.566364 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdea8fe-fe4f-4eec-aad8-f475659fbfe6 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:48.751365 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:48.767163 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffca204-1466-4019-b8ea-5861196d6189 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:48.897493 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a251b65b-533b-4d6f-b6d6-e4a02dd33ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:48.902979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251b65b-533b-4d6f-b6d6-e4a02dd33ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:48.907422 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251b65b-533b-4d6f-b6d6-e4a02dd33ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:48.908201 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251b65b-533b-4d6f-b6d6-e4a02dd33ee6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:48.951716 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdea8fe-fe4f-4eec-aad8-f475659fbfe6 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:48.965065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:49.070101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a251b65b-533b-4d6f-b6d6-e4a02dd33ee6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:49.076562 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a251b65b-533b-4d6f-b6d6-e4a02dd33ee6 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:49.093291 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343219c-c694-4158-9c88-359532e6fe53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:49.095606 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343219c-c694-4158-9c88-359532e6fe53 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:49.098116 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343219c-c694-4158-9c88-359532e6fe53 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:49.105163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343219c-c694-4158-9c88-359532e6fe53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:49.170239 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343219c-c694-4158-9c88-359532e6fe53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:49.172472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343219c-c694-4158-9c88-359532e6fe53 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:49.246320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-1343219c-c694-4158-9c88-359532e6fe53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:49.246320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:49.246320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:49.246320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:49.246320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:49.246320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:49.246320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:49.246320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:49.246320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:49.246320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:49.246320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:49.246320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:49.246320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:49.246320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:49.246320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:49.247275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:49.247275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:49.247275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:49.247275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:49.247275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:49.247275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:49.247275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:49.247275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:49.247275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:49.247275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:49.247275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:49.247275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:49.247275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:49.247275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:49.247275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:49.247275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:49.247275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:49.248215 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:49.248215 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:49.248215 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:49.248215 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:49.253623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 277/556] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 689 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:49.254855 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343219c-c694-4158-9c88-359532e6fe53 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:49.409072 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 280/557] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:30:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 937 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:30:49.409072 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a251b65b-533b-4d6f-b6d6-e4a02dd33ee6 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:49.477986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:49.482749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343219c-c694-4158-9c88-359532e6fe53 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:49.521242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:49.542070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb5cc9-b55e-4f3c-a746-d857b7bb6bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:49.543445 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb5cc9-b55e-4f3c-a746-d857b7bb6bb4 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:49.543445 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb5cc9-b55e-4f3c-a746-d857b7bb6bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:49.543898 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb5cc9-b55e-4f3c-a746-d857b7bb6bb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:49.561334 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadb5cc9-b55e-4f3c-a746-d857b7bb6bb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:49.574152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-dadb5cc9-b55e-4f3c-a746-d857b7bb6bb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:49.574152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:49.574152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:49.574152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:49.574152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:49.574152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:49.574152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:49.574152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:49.574152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:49.574152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:49.574152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:49.574152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:49.574152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:49.574152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:49.574152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:49.574152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:49.575911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:49.575911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:49.575911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:49.575911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:49.575911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:49.575911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:49.575911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:49.575911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:49.575911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:49.575911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:49.575911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:49.575911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:49.575911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:49.575911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:49.575911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:49.575911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:49.577768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:49.577768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:49.577768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:49.577768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:49.577768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:49.577768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:49.577768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:49.577768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:49.577768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:49.577768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:49.577768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:49.577768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:49.577768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 278/558] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:30:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:49.579223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb3c5d-4fc0-45f4-aec7-fd4866253ca6 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:49.579223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb3c5d-4fc0-45f4-aec7-fd4866253ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:49.579496 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb3c5d-4fc0-45f4-aec7-fd4866253ca6 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:49.582040 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 279/559] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:49.583206 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb3c5d-4fc0-45f4-aec7-fd4866253ca6 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:49.606134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a251b65b-533b-4d6f-b6d6-e4a02dd33ee6 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:49.651566 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcb40d0-3b07-4a30-9c04-a0bbcbff7c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:49.652433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb40d0-3b07-4a30-9c04-a0bbcbff7c4e None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:49.652901 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb40d0-3b07-4a30-9c04-a0bbcbff7c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:49.653292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcb40d0-3b07-4a30-9c04-a0bbcbff7c4e None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:49.655033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb40d0-3b07-4a30-9c04-a0bbcbff7c4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:49.655033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb40d0-3b07-4a30-9c04-a0bbcbff7c4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:49.657028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcb40d0-3b07-4a30-9c04-a0bbcbff7c4e None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:49.681447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 281/560] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:49.681447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cd0cf-3bd0-4be6-bc04-06e570cfd138 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:49.681447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cd0cf-3bd0-4be6-bc04-06e570cfd138 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:49.681447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2cd0cf-3bd0-4be6-bc04-06e570cfd138 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:49.681447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 282/561] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:49.681447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cd0cf-3bd0-4be6-bc04-06e570cfd138 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:49.721129 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb3c5d-4fc0-45f4-aec7-fd4866253ca6 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:49.768667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d95a8-5bca-401b-80ea-36b2a7b515b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:49.769517 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d95a8-5bca-401b-80ea-36b2a7b515b1 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:49.769955 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d95a8-5bca-401b-80ea-36b2a7b515b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:49.770239 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d95a8-5bca-401b-80ea-36b2a7b515b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:49.787946 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d95a8-5bca-401b-80ea-36b2a7b515b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:49.789644 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d95a8-5bca-401b-80ea-36b2a7b515b1 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:49.811787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-497d95a8-5bca-401b-80ea-36b2a7b515b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:49.811787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:49.811787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:49.811787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:49.811787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:49.811787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:49.811787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:49.811787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:49.811787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:49.811787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:49.811787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:49.811787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:49.811787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:49.811787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:49.811787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:49.812922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:49.812922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:49.812922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:49.812922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:49.812922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:49.812922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:49.812922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:49.812922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:49.812922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:49.812922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:49.812922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:49.812922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:49.812922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:49.812922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:49.812922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:49.812922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:49.812922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:49.814461 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:49.814461 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:49.814461 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:49.814461 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:49.814461 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 280/562] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:49.814461 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98d6f9-74dd-47e4-9b9a-143bbdac7585 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:49.815022 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98d6f9-74dd-47e4-9b9a-143bbdac7585 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:49.815022 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de98d6f9-74dd-47e4-9b9a-143bbdac7585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:49.832368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2cd0cf-3bd0-4be6-bc04-06e570cfd138 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:49.832872 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-de98d6f9-74dd-47e4-9b9a-143bbdac7585 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:49.834140 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-de98d6f9-74dd-47e4-9b9a-143bbdac7585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:49.835159 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:49.843657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:49.877855 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0da44a-b9a2-4fcb-b08a-dcabefb3b108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:49.879053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0da44a-b9a2-4fcb-b08a-dcabefb3b108 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:49.879267 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0da44a-b9a2-4fcb-b08a-dcabefb3b108 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:49.879711 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0da44a-b9a2-4fcb-b08a-dcabefb3b108 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:49.881104 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0da44a-b9a2-4fcb-b08a-dcabefb3b108 None admin] RBAC: Adding query filter params (name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:49.881519 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0da44a-b9a2-4fcb-b08a-dcabefb3b108 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:49.884252 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0da44a-b9a2-4fcb-b08a-dcabefb3b108 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:49.905649 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 283/563] 10.222.0.13 () {56 vars in 1103 bytes} [Fri May 27 19:30:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:49.905649 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0da44a-b9a2-4fcb-b08a-dcabefb3b108 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:49.927604 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 281/564] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:49.930339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ede414-d628-464e-8fa9-72e225c8543e None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:49.931288 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ede414-d628-464e-8fa9-72e225c8543e None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:49.931288 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ede414-d628-464e-8fa9-72e225c8543e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:49.963320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-b9ede414-d628-464e-8fa9-72e225c8543e None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:49.965170 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-b9ede414-d628-464e-8fa9-72e225c8543e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:50.051544 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0da44a-b9a2-4fcb-b08a-dcabefb3b108 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:50.060398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 282/565] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:50.061305 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ede414-d628-464e-8fa9-72e225c8543e None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:50.123383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978ace5-7328-4e55-901e-1c78e05bdd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:50.124640 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978ace5-7328-4e55-901e-1c78e05bdd17 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:50.125154 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978ace5-7328-4e55-901e-1c78e05bdd17 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:50.125595 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978ace5-7328-4e55-901e-1c78e05bdd17 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:50.139274 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2978ace5-7328-4e55-901e-1c78e05bdd17 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:50.141814 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2978ace5-7328-4e55-901e-1c78e05bdd17 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:50.166568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-2978ace5-7328-4e55-901e-1c78e05bdd17 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 19:30:50.166568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:50.166568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:50.166568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:50.166568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:50.166568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:50.166568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:50.166568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:50.166568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:50.166568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:50.166568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:50.166568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:50.166568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:30:50.166568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:30:50.166568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:30:50.166568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:30:50.168145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:50.168145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:50.168145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:30:50.168145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:50.168145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:30:50.168145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:50.168145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:50.168145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:50.168145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:50.168145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:50.168145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:50.168145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:50.168145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:50.168145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:50.168145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:50.168145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:50.168145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:50.168145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:50.169622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:50.169622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:50.169622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:30:50.169622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:30:50.169622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:30:50.169622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:30:50.169622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:30:50.169622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:30:50.169622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 19:30:50.169622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:50.170659 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:50.171139 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 284/566] 10.222.0.13 () {56 vars in 1093 bytes} [Fri May 27 19:30:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:50.171139 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978ace5-7328-4e55-901e-1c78e05bdd17 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:50.218131 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:50.224620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ede414-d628-464e-8fa9-72e225c8543e None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:50.262843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72102c14-ffbd-4c0f-9152-fd8cf18e9744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:50.263775 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72102c14-ffbd-4c0f-9152-fd8cf18e9744 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:50.264074 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72102c14-ffbd-4c0f-9152-fd8cf18e9744 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:50.264331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72102c14-ffbd-4c0f-9152-fd8cf18e9744 None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:50.265160 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72102c14-ffbd-4c0f-9152-fd8cf18e9744 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:50.265464 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72102c14-ffbd-4c0f-9152-fd8cf18e9744 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:50.266899 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72102c14-ffbd-4c0f-9152-fd8cf18e9744 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:50.280713 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 283/567] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:50.281642 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72102c14-ffbd-4c0f-9152-fd8cf18e9744 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:50.329153 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978ace5-7328-4e55-901e-1c78e05bdd17 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:50.377684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e9454-c8f3-4b47-b3b2-1d6091ad099d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:50.378870 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e9454-c8f3-4b47-b3b2-1d6091ad099d None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:50.379290 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e9454-c8f3-4b47-b3b2-1d6091ad099d None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:50.379703 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e9454-c8f3-4b47-b3b2-1d6091ad099d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:50.398323 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e9454-c8f3-4b47-b3b2-1d6091ad099d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:50.400843 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e9454-c8f3-4b47-b3b2-1d6091ad099d None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:50.422092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-f73e9454-c8f3-4b47-b3b2-1d6091ad099d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:50.422092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:50.422092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:50.422092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:50.422092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:50.422092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:50.422092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:50.422092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:50.422092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:50.422092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:50.422092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:50.422092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:50.422092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:50.422092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:50.422092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:50.424180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:50.424180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:50.424180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:50.424180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:50.424180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:50.424180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:50.424180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:50.424180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:50.424180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:50.424180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:50.424180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:50.424180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:50.424180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:50.424180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:50.424180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:50.424180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:50.424180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:50.426247 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:50.426247 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:50.426247 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:50.426247 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:50.426247 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 285/568] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:50.426247 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e9454-c8f3-4b47-b3b2-1d6091ad099d None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:50.435261 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72102c14-ffbd-4c0f-9152-fd8cf18e9744 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:50.474857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:50.482416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc069e1-f58f-4df7-82f2-542b614c09a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:50.483363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc069e1-f58f-4df7-82f2-542b614c09a1 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:50.483710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc069e1-f58f-4df7-82f2-542b614c09a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:50.484031 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc069e1-f58f-4df7-82f2-542b614c09a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:50.499795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc069e1-f58f-4df7-82f2-542b614c09a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:50.509715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-adc069e1-f58f-4df7-82f2-542b614c09a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:50.509715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:50.509715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:50.509715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:50.509715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:50.509715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:50.509715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:50.509715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:50.509715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:50.509715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:50.509715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:50.509715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:50.509715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:50.509715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:50.509715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:50.509715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:50.511555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:50.511555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:50.511555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:50.511555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:50.511555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:50.511555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:50.511555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:50.511555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:50.511555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:50.511555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:50.511555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:50.511555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:50.511555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:50.511555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:50.511555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:50.511555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:50.516080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:50.516080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:50.516080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:50.516080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:50.516080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:50.516080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:50.516080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:50.516080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:50.516080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:50.516080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:50.516080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:50.516080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:50.516080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 284/569] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:30:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:50.516080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc069e1-f58f-4df7-82f2-542b614c09a1 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:50.552832 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:50.579601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e9454-c8f3-4b47-b3b2-1d6091ad099d None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:50.635203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0d3b9-e7f3-4e6b-b6d5-e4271a0f7d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:50.636317 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0d3b9-e7f3-4e6b-b6d5-e4271a0f7d77 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:50.636703 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0d3b9-e7f3-4e6b-b6d5-e4271a0f7d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:50.637315 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0d3b9-e7f3-4e6b-b6d5-e4271a0f7d77 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:50.653678 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0d3b9-e7f3-4e6b-b6d5-e4271a0f7d77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:50.664023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-dbd0d3b9-e7f3-4e6b-b6d5-e4271a0f7d77 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:50.664023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:50.664023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:50.664023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:50.664023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:50.664023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:50.664023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:50.664023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:50.664023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:50.664023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:50.664023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:50.664023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:50.664023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:50.664023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:50.664023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:50.664023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:50.665924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:50.665924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:50.665924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:50.665924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:50.665924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:50.665924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:50.665924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:50.665924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:50.665924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:50.665924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:50.665924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:50.665924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:50.665924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:50.665924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:50.665924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:50.665924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:50.667571 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:50.667571 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:50.667571 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:50.667571 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:50.667571 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:50.667571 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:50.667571 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:50.667571 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:50.667571 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:50.667571 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:50.667571 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:50.667571 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:50.667571 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 286/570] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:30:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:50.667571 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0d3b9-e7f3-4e6b-b6d5-e4271a0f7d77 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:50.669048 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc069e1-f58f-4df7-82f2-542b614c09a1 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:50.708045 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188f569-e818-4583-b19c-9d7f04424adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:50.708715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188f569-e818-4583-b19c-9d7f04424adf None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:50.709016 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188f569-e818-4583-b19c-9d7f04424adf None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:50.709297 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188f569-e818-4583-b19c-9d7f04424adf None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:50.710675 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188f569-e818-4583-b19c-9d7f04424adf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:50.710993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188f569-e818-4583-b19c-9d7f04424adf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:50.712581 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3188f569-e818-4583-b19c-9d7f04424adf None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:50.738758 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 285/571] 10.222.0.13 () {56 vars in 1133 bytes} [Fri May 27 19:30:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:50.738758 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188f569-e818-4583-b19c-9d7f04424adf None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:50.796629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:50.806377 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0d3b9-e7f3-4e6b-b6d5-e4271a0f7d77 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:50.850270 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ba20a-135a-4038-bf81-ff37b1508a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:50.853804 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ba20a-135a-4038-bf81-ff37b1508a94 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:50.853961 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3188f569-e818-4583-b19c-9d7f04424adf None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:50.854767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ba20a-135a-4038-bf81-ff37b1508a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:50.855056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ba20a-135a-4038-bf81-ff37b1508a94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:50.865540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ba20a-135a-4038-bf81-ff37b1508a94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:50.867795 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ba20a-135a-4038-bf81-ff37b1508a94 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:50.869852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:50.885061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-089ba20a-135a-4038-bf81-ff37b1508a94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:30:50.885061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:50.885061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:50.885061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:50.885061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:50.885061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:50.885061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:50.885061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:50.885061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:50.885061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:50.885061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:50.885061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:50.885061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:30:50.885061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:30:50.885061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:30:50.886075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:30:50.886075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:50.886075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:50.886075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:50.886075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:50.886075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:50.886075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:50.886075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:50.886075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:50.886075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:50.886075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:50.886075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:50.886075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:50.886075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:50.886075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:50.886075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:50.886075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:50.886967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:30:50.886967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:30:50.886967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:30:50.886967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:30:50.886967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:30:50.886967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:30:50.886967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:30:50.886967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:50.886967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 287/572] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:30:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:50.888044 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ba20a-135a-4038-bf81-ff37b1508a94 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:50.898628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae62af3-b6c3-4f1f-b322-654ffb5b6fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:50.899488 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae62af3-b6c3-4f1f-b322-654ffb5b6fba None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:50.900150 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae62af3-b6c3-4f1f-b322-654ffb5b6fba None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:50.900150 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae62af3-b6c3-4f1f-b322-654ffb5b6fba None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:50.900570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae62af3-b6c3-4f1f-b322-654ffb5b6fba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:50.900879 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae62af3-b6c3-4f1f-b322-654ffb5b6fba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:50.902341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae62af3-b6c3-4f1f-b322-654ffb5b6fba None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:50.916408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 286/573] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:50.917580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae62af3-b6c3-4f1f-b322-654ffb5b6fba None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:51.014328 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ba20a-135a-4038-bf81-ff37b1508a94 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:51.037301 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae62af3-b6c3-4f1f-b322-654ffb5b6fba None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:51.050361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:51.057056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d96d02-3bda-42e3-ae8c-ffcc14839507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:51.057766 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d96d02-3bda-42e3-ae8c-ffcc14839507 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:51.058013 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d96d02-3bda-42e3-ae8c-ffcc14839507 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:51.058283 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d96d02-3bda-42e3-ae8c-ffcc14839507 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:51.059039 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d96d02-3bda-42e3-ae8c-ffcc14839507 None admin] RBAC: Adding query filter params (name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:51.059338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d96d02-3bda-42e3-ae8c-ffcc14839507 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:51.060778 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d96d02-3bda-42e3-ae8c-ffcc14839507 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:51.071290 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 288/574] 10.222.0.13 () {56 vars in 1103 bytes} [Fri May 27 19:30:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:51.072892 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d96d02-3bda-42e3-ae8c-ffcc14839507 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:51.089771 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9f065-0d8c-450e-8db0-d728ed8fd31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:51.090456 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9f065-0d8c-450e-8db0-d728ed8fd31c None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:51.090750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9f065-0d8c-450e-8db0-d728ed8fd31c None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:51.091030 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9f065-0d8c-450e-8db0-d728ed8fd31c None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:51.091788 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d9f065-0d8c-450e-8db0-d728ed8fd31c None admin] RBAC: Adding query filter params (name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:51.092090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d9f065-0d8c-450e-8db0-d728ed8fd31c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:51.093578 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d9f065-0d8c-450e-8db0-d728ed8fd31c None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:51.106426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 287/575] 10.222.0.13 () {56 vars in 1103 bytes} [Fri May 27 19:30:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:51.107286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9f065-0d8c-450e-8db0-d728ed8fd31c None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:51.153871 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:51.201643 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d96d02-3bda-42e3-ae8c-ffcc14839507 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:51.229361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9f065-0d8c-450e-8db0-d728ed8fd31c None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:51.244169 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc8649-123e-4bd9-99c8-fc3636c05db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:51.244995 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc8649-123e-4bd9-99c8-fc3636c05db3 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:51.245289 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc8649-123e-4bd9-99c8-fc3636c05db3 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:51.245563 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc8649-123e-4bd9-99c8-fc3636c05db3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:51.256695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc8649-123e-4bd9-99c8-fc3636c05db3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:51.258376 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfc8649-123e-4bd9-99c8-fc3636c05db3 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:51.269513 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627237f-ab63-4ea8-9a92-139040895571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:51.270109 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627237f-ab63-4ea8-9a92-139040895571 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:51.270389 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627237f-ab63-4ea8-9a92-139040895571 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:51.270667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627237f-ab63-4ea8-9a92-139040895571 None admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:51.271451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627237f-ab63-4ea8-9a92-139040895571 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:51.271755 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627237f-ab63-4ea8-9a92-139040895571 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:51.273717 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f627237f-ab63-4ea8-9a92-139040895571 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:51.277120 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-dcfc8649-123e-4bd9-99c8-fc3636c05db3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:51.277120 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:51.277120 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:51.277120 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:51.277120 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:51.277120 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:51.277120 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:51.277120 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:51.277120 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:51.277120 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:51.277120 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:51.277120 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:51.277120 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:51.277120 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:51.277120 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:51.284593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:51.284593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:51.284593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:51.284593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:51.284593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:51.284593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:51.284593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:51.284593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:51.284593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:51.284593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:51.284593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:51.284593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:51.284593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:51.284593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:51.284593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:51.284593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:51.284593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:51.285548 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:51.285548 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:51.285548 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:51.285548 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:51.285548 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 289/576] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:51.285548 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc8649-123e-4bd9-99c8-fc3636c05db3 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:51.291807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 288/577] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:30:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:51.292676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627237f-ab63-4ea8-9a92-139040895571 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:51.331342 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:51.402906 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc8649-123e-4bd9-99c8-fc3636c05db3 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:51.416406 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627237f-ab63-4ea8-9a92-139040895571 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:51.418408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:51.446424 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0356d6-171e-4839-ba16-63542c82bde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:51.447427 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0356d6-171e-4839-ba16-63542c82bde8 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:51.447771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0356d6-171e-4839-ba16-63542c82bde8 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:51.448120 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0356d6-171e-4839-ba16-63542c82bde8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:51.457180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0356d6-171e-4839-ba16-63542c82bde8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:51.458677 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0356d6-171e-4839-ba16-63542c82bde8 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:51.461495 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ffbbb-a72f-4f39-9978-3b579194ab6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:51.462300 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ffbbb-a72f-4f39-9978-3b579194ab6a None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:51.462588 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ffbbb-a72f-4f39-9978-3b579194ab6a None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:51.462863 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ffbbb-a72f-4f39-9978-3b579194ab6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:51.475466 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ffbbb-a72f-4f39-9978-3b579194ab6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:51.478318 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ffbbb-a72f-4f39-9978-3b579194ab6a None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:51.480204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-aa0356d6-171e-4839-ba16-63542c82bde8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 19:30:51.480204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:51.480204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:51.480204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:51.480204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:51.480204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:51.480204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:51.480204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:51.480204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:51.480204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:51.480204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:51.480204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:51.480204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:30:51.480204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:30:51.480204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:30:51.480204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:30:51.481530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:51.481530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:51.481530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:30:51.481530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:51.481530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:30:51.481530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:51.481530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:51.481530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:51.481530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:51.481530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:51.481530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:51.481530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:51.481530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:51.481530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:51.481530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:51.481530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:51.481530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:51.481530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:51.482572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:51.482572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:51.482572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:30:51.482572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:30:51.482572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:30:51.482572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:30:51.482572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:30:51.482572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:30:51.482572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 19:30:51.482572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:51.482572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 290/578] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:51.482572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0356d6-171e-4839-ba16-63542c82bde8 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:51.504981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-f35ffbbb-a72f-4f39-9978-3b579194ab6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:51.504981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:51.504981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:51.504981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:51.504981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:51.504981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:51.504981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:51.504981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:51.504981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:51.504981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:51.504981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:51.504981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:51.504981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:51.504981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:51.504981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:51.506589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:51.506589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:51.506589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:51.506589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:51.506589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:51.506589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:51.506589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:51.506589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:51.506589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:51.506589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:51.506589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:51.506589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:51.506589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:51.506589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:51.506589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:51.506589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:51.506589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:51.507893 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:51.507893 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:51.507893 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:51.507893 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:51.509417 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 289/579] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:51.510152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ffbbb-a72f-4f39-9978-3b579194ab6a None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:51.621283 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0356d6-171e-4839-ba16-63542c82bde8 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:51.625701 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:51.646921 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ffbbb-a72f-4f39-9978-3b579194ab6a None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:51.665048 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0bea8-8bb6-41c1-b20e-7e9f4a35c1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:51.666072 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0bea8-8bb6-41c1-b20e-7e9f4a35c1b8 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:51.666364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0bea8-8bb6-41c1-b20e-7e9f4a35c1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:51.666645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0bea8-8bb6-41c1-b20e-7e9f4a35c1b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:51.683231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0bea8-8bb6-41c1-b20e-7e9f4a35c1b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:51.684747 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0bea8-8bb6-41c1-b20e-7e9f4a35c1b8 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:51.689038 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca7277-b2c7-4cb2-9d7e-98a3b97260f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:51.689589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca7277-b2c7-4cb2-9d7e-98a3b97260f4 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:51.690132 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca7277-b2c7-4cb2-9d7e-98a3b97260f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:51.690251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca7277-b2c7-4cb2-9d7e-98a3b97260f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:51.691189 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ca7277-b2c7-4cb2-9d7e-98a3b97260f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:51.691690 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ca7277-b2c7-4cb2-9d7e-98a3b97260f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:51.693288 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ca7277-b2c7-4cb2-9d7e-98a3b97260f4 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:51.703601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-2ae0bea8-8bb6-41c1-b20e-7e9f4a35c1b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:51.703601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:51.703601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:51.703601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:51.703601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:51.703601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:51.703601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:51.703601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:51.703601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:51.703601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:51.703601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:51.703601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:51.703601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:51.703601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:51.703601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:51.710700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:51.710700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:51.710700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:51.710700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:51.710700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:51.710700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:51.710700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:51.710700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:51.710700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:51.710700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:51.710700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:51.710700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:51.710700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:51.710700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:51.710700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:51.710700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:51.710700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:51.711621 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:51.711621 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:51.711621 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:51.711621 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:51.711621 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 291/580] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:51.711621 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0bea8-8bb6-41c1-b20e-7e9f4a35c1b8 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:51.715297 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 290/581] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:51.716176 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca7277-b2c7-4cb2-9d7e-98a3b97260f4 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:51.745231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:51.844081 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca7277-b2c7-4cb2-9d7e-98a3b97260f4 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:51.853367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0bea8-8bb6-41c1-b20e-7e9f4a35c1b8 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:51.896095 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf703d94-af4d-419b-a7a2-1eb91184deb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:51.896674 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf703d94-af4d-419b-a7a2-1eb91184deb8 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:51.896969 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf703d94-af4d-419b-a7a2-1eb91184deb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:51.898885 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf703d94-af4d-419b-a7a2-1eb91184deb8 None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:51.898885 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf703d94-af4d-419b-a7a2-1eb91184deb8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:51.898885 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf703d94-af4d-419b-a7a2-1eb91184deb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:51.899359 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340d936-3b0b-4818-adf8-25a967c51414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:51.899536 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340d936-3b0b-4818-adf8-25a967c51414 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:51.899778 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340d936-3b0b-4818-adf8-25a967c51414 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:51.900058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340d936-3b0b-4818-adf8-25a967c51414 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:51.900906 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340d936-3b0b-4818-adf8-25a967c51414 None admin] RBAC: Adding query filter params (role.id=11e7404599ae49afb5edd22dabe1a5b2, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=37bc6d0abb454de19e1ab2f46bf4a593) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:51.901253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340d936-3b0b-4818-adf8-25a967c51414 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:51.901409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf703d94-af4d-419b-a7a2-1eb91184deb8 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:51.902719 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a340d936-3b0b-4818-adf8-25a967c51414 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:51.909905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 292/582] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:30:51 2022] GET /identity/v3/role_assignments?user.id=37bc6d0abb454de19e1ab2f46bf4a593&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=11e7404599ae49afb5edd22dabe1a5b2 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:51.911107 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340d936-3b0b-4818-adf8-25a967c51414 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:51.938242 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 291/583] 10.222.0.13 () {56 vars in 1137 bytes} [Fri May 27 19:30:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:51.939178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf703d94-af4d-419b-a7a2-1eb91184deb8 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:51.942250 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:52.060301 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:52.077660 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a340d936-3b0b-4818-adf8-25a967c51414 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:52.101910 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf703d94-af4d-419b-a7a2-1eb91184deb8 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:52.131791 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2442fcc-ac7e-4daf-8150-9864051ba74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:52.134014 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2442fcc-ac7e-4daf-8150-9864051ba74d None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:52.134606 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2442fcc-ac7e-4daf-8150-9864051ba74d None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:52.134983 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2442fcc-ac7e-4daf-8150-9864051ba74d None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:52.135966 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2442fcc-ac7e-4daf-8150-9864051ba74d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:52.136682 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2442fcc-ac7e-4daf-8150-9864051ba74d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:52.140011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2442fcc-ac7e-4daf-8150-9864051ba74d None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:52.161058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 293/584] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:52.161058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2442fcc-ac7e-4daf-8150-9864051ba74d None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:52.167019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c339ef-dcb0-4d97-b26f-f097bc069fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:52.167852 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c339ef-dcb0-4d97-b26f-f097bc069fdf None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:52.168338 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c339ef-dcb0-4d97-b26f-f097bc069fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:52.168824 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c339ef-dcb0-4d97-b26f-f097bc069fdf None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:52.170234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c339ef-dcb0-4d97-b26f-f097bc069fdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:52.170786 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c339ef-dcb0-4d97-b26f-f097bc069fdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:52.172854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c339ef-dcb0-4d97-b26f-f097bc069fdf None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:52.191315 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 292/585] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:52.192295 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c339ef-dcb0-4d97-b26f-f097bc069fdf None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:52.282013 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:52.325498 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2442fcc-ac7e-4daf-8150-9864051ba74d None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:52.334093 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c339ef-dcb0-4d97-b26f-f097bc069fdf None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:52.373141 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7bcc55-8a75-4be0-b0df-b14a3206e0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:52.373949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bcc55-8a75-4be0-b0df-b14a3206e0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:52.374235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bcc55-8a75-4be0-b0df-b14a3206e0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:52.374514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bcc55-8a75-4be0-b0df-b14a3206e0e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:52.379309 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7575603-b580-4edf-9d02-29b893b2db47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:52.380420 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7575603-b580-4edf-9d02-29b893b2db47 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:52.381163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7575603-b580-4edf-9d02-29b893b2db47 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:52.381483 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7575603-b580-4edf-9d02-29b893b2db47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:52.382466 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7bcc55-8a75-4be0-b0df-b14a3206e0e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:52.383816 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7bcc55-8a75-4be0-b0df-b14a3206e0e7 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:52.399035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-1d7bcc55-8a75-4be0-b0df-b14a3206e0e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:30:52.399035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:52.399035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:52.399035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:52.399035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:52.399035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:52.399035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:52.399035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:52.399035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:52.399035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:52.399035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:52.399035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:52.399035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:30:52.399035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:30:52.399035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:30:52.407333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:30:52.407333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:52.407333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:52.407333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:52.407333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:52.407333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:52.407333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:52.407333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:52.407333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:52.407333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:52.407333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:52.407333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:52.407333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:52.407333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:52.407333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:52.407333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:52.407333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:52.408593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:30:52.408593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:30:52.408593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:30:52.408593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:30:52.408593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:30:52.408593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:30:52.408593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:30:52.408593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:52.409314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7575603-b580-4edf-9d02-29b893b2db47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:52.409393 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 294/586] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:30:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:52.409515 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7575603-b580-4edf-9d02-29b893b2db47 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:52.409647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7bcc55-8a75-4be0-b0df-b14a3206e0e7 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:52.422297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:52.431191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-f7575603-b580-4edf-9d02-29b893b2db47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:52.431191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:52.431191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:52.431191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:52.431191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:52.431191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:52.431191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:52.431191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:52.431191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:52.431191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:52.431191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:52.431191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:52.431191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:52.431191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:52.431191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:52.440362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:52.440362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:52.440362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:52.440362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:52.440362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:52.440362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:52.440362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:52.440362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:52.440362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:52.440362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:52.440362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:52.440362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:52.440362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:52.440362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:52.440362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:52.440362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:52.440362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:52.441282 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:52.441282 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:52.441282 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:52.441282 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:52.441282 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 293/587] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:52.441282 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7575603-b580-4edf-9d02-29b893b2db47 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:52.550897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7bcc55-8a75-4be0-b0df-b14a3206e0e7 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:52.596922 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5359f-705e-493b-a8a6-62664654f773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:52.598001 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5359f-705e-493b-a8a6-62664654f773 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:52.598451 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5359f-705e-493b-a8a6-62664654f773 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:52.598926 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5359f-705e-493b-a8a6-62664654f773 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:52.600236 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7575603-b580-4edf-9d02-29b893b2db47 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:52.607808 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe5359f-705e-493b-a8a6-62664654f773 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:52.609471 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe5359f-705e-493b-a8a6-62664654f773 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:52.618396 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:52.624930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-0fe5359f-705e-493b-a8a6-62664654f773 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 19:30:52.624930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:52.624930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:52.624930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:52.624930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:52.624930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:52.624930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:52.624930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:52.624930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:52.624930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:52.624930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:52.624930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:52.624930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:30:52.624930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:30:52.624930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:30:52.624930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:30:52.625998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:52.625998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:52.625998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:30:52.625998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:52.625998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:30:52.625998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:52.625998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:52.625998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:52.625998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:52.625998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:52.625998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:52.625998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:52.625998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:52.625998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:52.625998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:52.625998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:52.625998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:52.625998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:52.626955 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:52.626955 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:52.626955 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:30:52.626955 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:30:52.626955 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:30:52.626955 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:30:52.626955 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:30:52.626955 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:30:52.626955 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 19:30:52.626955 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:52.626955 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 295/588] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:30:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:52.627628 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5359f-705e-493b-a8a6-62664654f773 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:52.645078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152db4a-8c9d-4e07-ac5a-1df295bfd1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:52.646001 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152db4a-8c9d-4e07-ac5a-1df295bfd1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:52.646241 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152db4a-8c9d-4e07-ac5a-1df295bfd1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:52.646516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152db4a-8c9d-4e07-ac5a-1df295bfd1a8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:52.656049 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152db4a-8c9d-4e07-ac5a-1df295bfd1a8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:52.657661 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152db4a-8c9d-4e07-ac5a-1df295bfd1a8 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:52.681065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-a152db4a-8c9d-4e07-ac5a-1df295bfd1a8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 19:30:52.681065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:52.681065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:52.681065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:52.681065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:52.681065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:52.681065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:52.681065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:52.681065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:52.681065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:52.681065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:52.681065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:52.681065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:30:52.681065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:30:52.681065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:30:52.681065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:30:52.682455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:52.682455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:52.682455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:30:52.682455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:52.682455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:30:52.682455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:52.682455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:52.682455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:52.682455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:52.682455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:52.682455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:52.682455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:52.682455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:52.682455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:52.682455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:52.682455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:52.682455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:52.682455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:52.683887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:52.683887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:52.683887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:30:52.683887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:30:52.683887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:30:52.683887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:30:52.683887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:30:52.683887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:30:52.683887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 19:30:52.683887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:52.693124 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 294/589] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:52.693124 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152db4a-8c9d-4e07-ac5a-1df295bfd1a8 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:52.724127 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:52.767462 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5359f-705e-493b-a8a6-62664654f773 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:52.820692 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d572b5-59ec-4249-9680-845379473999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:52.821719 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d572b5-59ec-4249-9680-845379473999 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:52.822085 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d572b5-59ec-4249-9680-845379473999 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:52.822522 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d572b5-59ec-4249-9680-845379473999 None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:52.823580 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d572b5-59ec-4249-9680-845379473999 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:52.824112 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d572b5-59ec-4249-9680-845379473999 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:52.824484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152db4a-8c9d-4e07-ac5a-1df295bfd1a8 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:52.826300 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d572b5-59ec-4249-9680-845379473999 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:52.849474 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 296/590] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:30:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:52.849474 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d572b5-59ec-4249-9680-845379473999 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:52.868997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b0f79-7252-494b-9f3f-bf96a7b3be1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:52.869674 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b0f79-7252-494b-9f3f-bf96a7b3be1f None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:52.869961 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b0f79-7252-494b-9f3f-bf96a7b3be1f None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:52.870248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b0f79-7252-494b-9f3f-bf96a7b3be1f None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:52.870968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b0f79-7252-494b-9f3f-bf96a7b3be1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:52.871289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b0f79-7252-494b-9f3f-bf96a7b3be1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:52.872698 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b0f79-7252-494b-9f3f-bf96a7b3be1f None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:52.891933 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 295/591] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:52.891933 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b0f79-7252-494b-9f3f-bf96a7b3be1f None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:52.932799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:52.988166 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d572b5-59ec-4249-9680-845379473999 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:53.021803 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b0f79-7252-494b-9f3f-bf96a7b3be1f None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:53.030859 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:53.040868 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7549b-9e72-4a9b-b899-af5f7884e564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:53.041850 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7549b-9e72-4a9b-b899-af5f7884e564 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:53.042145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7549b-9e72-4a9b-b899-af5f7884e564 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:53.042609 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b7549b-9e72-4a9b-b899-af5f7884e564 None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:53.043654 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7549b-9e72-4a9b-b899-af5f7884e564 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:53.043929 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7549b-9e72-4a9b-b899-af5f7884e564 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:53.045777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b7549b-9e72-4a9b-b899-af5f7884e564 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:53.061403 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4b1f8-0187-4185-b0dc-1e46a72c2c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:53.062087 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4b1f8-0187-4185-b0dc-1e46a72c2c68 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:53.062564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4b1f8-0187-4185-b0dc-1e46a72c2c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:53.062865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b4b1f8-0187-4185-b0dc-1e46a72c2c68 None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:53.064067 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b4b1f8-0187-4185-b0dc-1e46a72c2c68 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:53.064313 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b4b1f8-0187-4185-b0dc-1e46a72c2c68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:53.066114 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b4b1f8-0187-4185-b0dc-1e46a72c2c68 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:53.070498 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 297/592] 10.222.0.13 () {56 vars in 1139 bytes} [Fri May 27 19:30:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:53.071926 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7549b-9e72-4a9b-b899-af5f7884e564 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:53.088387 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 296/593] 10.222.0.13 () {56 vars in 1137 bytes} [Fri May 27 19:30:52 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:53.089041 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4b1f8-0187-4185-b0dc-1e46a72c2c68 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:53.201213 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b7549b-9e72-4a9b-b899-af5f7884e564 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:53.217158 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:53.229809 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b4b1f8-0187-4185-b0dc-1e46a72c2c68 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:53.259488 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c44bc-b7b1-4e4b-87eb-e691b7d8a433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:53.261135 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c44bc-b7b1-4e4b-87eb-e691b7d8a433 None admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:53.261710 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c44bc-b7b1-4e4b-87eb-e691b7d8a433 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:53.262267 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c44bc-b7b1-4e4b-87eb-e691b7d8a433 None admin] PATH_INFO: `/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/7542e1fd160e4b2c9d25accf19aad19f/roles/11e7404599ae49afb5edd22dabe1a5b2` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:53.289951 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c44bc-b7b1-4e4b-87eb-e691b7d8a433 None admin] RBAC: Authorizing `identity:create_grant(project_id=a47d6b6971c3452e9799e8146856bdc5, user_id=7542e1fd160e4b2c9d25accf19aad19f, role_id=11e7404599ae49afb5edd22dabe1a5b2)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:53.292710 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2c44bc-b7b1-4e4b-87eb-e691b7d8a433 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:53.294126 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e8087-eb38-4ca3-9468-845c2eb7ce0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:53.295581 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e8087-eb38-4ca3-9468-845c2eb7ce0a None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:53.296158 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e8087-eb38-4ca3-9468-845c2eb7ce0a None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:53.296646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e8087-eb38-4ca3-9468-845c2eb7ce0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:53.305828 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e8087-eb38-4ca3-9468-845c2eb7ce0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:53.307516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e8087-eb38-4ca3-9468-845c2eb7ce0a None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:53.323563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-613e8087-eb38-4ca3-9468-845c2eb7ce0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:30:53.323563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:53.323563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:53.323563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:53.323563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:53.323563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:53.323563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:53.323563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:53.323563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:53.323563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:53.323563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:53.323563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:53.323563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:30:53.323563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:30:53.323563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:30:53.325193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:30:53.325193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:53.325193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:53.325193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:53.325193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:53.325193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:53.325193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:53.325193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:53.325193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:53.325193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:53.325193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:53.325193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:53.325193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:53.325193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:53.325193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:53.325193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:53.325193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:53.326184 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:30:53.326184 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:30:53.326184 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:30:53.326184 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:30:53.326184 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:30:53.326184 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:30:53.326184 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:30:53.326184 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:53.326184 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 297/594] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:30:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:53.326184 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e8087-eb38-4ca3-9468-845c2eb7ce0a None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:53.392044 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 298/595] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:30:53 2022] PUT /identity/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/7542e1fd160e4b2c9d25accf19aad19f/roles/11e7404599ae49afb5edd22dabe1a5b2 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:30:53.393119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c44bc-b7b1-4e4b-87eb-e691b7d8a433 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:53.470524 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:53.498281 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e8087-eb38-4ca3-9468-845c2eb7ce0a None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:53.564585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c44bc-b7b1-4e4b-87eb-e691b7d8a433 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:53.575814 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5464f2-ee08-4a55-bb8e-b03fc8571909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:53.576945 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5464f2-ee08-4a55-bb8e-b03fc8571909 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:53.577540 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5464f2-ee08-4a55-bb8e-b03fc8571909 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:53.577903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5464f2-ee08-4a55-bb8e-b03fc8571909 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:53.596693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5464f2-ee08-4a55-bb8e-b03fc8571909 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:53.599054 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5464f2-ee08-4a55-bb8e-b03fc8571909 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:53.612212 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05127a15-fe85-49d9-b0a3-d6b0518f69b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:53.613434 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05127a15-fe85-49d9-b0a3-d6b0518f69b3 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:53.613932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05127a15-fe85-49d9-b0a3-d6b0518f69b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:53.614461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05127a15-fe85-49d9-b0a3-d6b0518f69b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:53.619667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-9d5464f2-ee08-4a55-bb8e-b03fc8571909 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:53.619667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:53.619667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:53.619667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:53.619667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:53.619667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:53.619667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:53.619667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:53.619667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:53.619667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:53.619667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:53.619667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:53.619667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:53.619667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:53.619667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:53.620625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:53.620625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:53.620625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:53.620625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:53.620625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:53.620625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:53.620625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:53.620625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:53.620625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:53.620625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:53.620625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:53.620625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:53.620625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:53.620625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:53.620625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:53.620625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:53.620625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:53.621570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:53.621570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:53.621570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:53.621570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:53.622221 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 298/596] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:53.622789 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5464f2-ee08-4a55-bb8e-b03fc8571909 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:53.626998 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:53.633524 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05127a15-fe85-49d9-b0a3-d6b0518f69b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:53.635659 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05127a15-fe85-49d9-b0a3-d6b0518f69b3 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:53.663458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-05127a15-fe85-49d9-b0a3-d6b0518f69b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:53.663458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:53.663458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:53.663458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:53.663458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:53.663458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:53.663458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:53.663458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:53.663458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:53.663458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:53.663458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:53.663458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:53.663458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:53.663458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:53.663458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:53.664428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:53.664428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:53.664428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:53.664428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:53.664428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:53.664428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:53.664428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:53.664428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:53.664428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:53.664428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:53.664428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:53.664428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:53.664428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:53.664428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:53.664428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:53.664428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:53.664428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:53.665743 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:53.665743 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:53.665743 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:53.665743 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:53.665743 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 299/597] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:53.666363 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05127a15-fe85-49d9-b0a3-d6b0518f69b3 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:53.767804 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5464f2-ee08-4a55-bb8e-b03fc8571909 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:53.799678 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:53.810585 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a42b9-9a83-4a3c-9a27-cc6729a3c590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:53.815816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a42b9-9a83-4a3c-9a27-cc6729a3c590 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:53.815816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a42b9-9a83-4a3c-9a27-cc6729a3c590 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:53.815816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a42b9-9a83-4a3c-9a27-cc6729a3c590 None admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:53.815816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a42b9-9a83-4a3c-9a27-cc6729a3c590 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:53.815816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a42b9-9a83-4a3c-9a27-cc6729a3c590 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:53.815816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937a42b9-9a83-4a3c-9a27-cc6729a3c590 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:53.816545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05127a15-fe85-49d9-b0a3-d6b0518f69b3 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:53.858664 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c6908-a4c6-4387-92ea-c61fd563cdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:53.859364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c6908-a4c6-4387-92ea-c61fd563cdda None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:53.859651 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c6908-a4c6-4387-92ea-c61fd563cdda None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:53.859928 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6c6908-a4c6-4387-92ea-c61fd563cdda None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:53.860684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c6908-a4c6-4387-92ea-c61fd563cdda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:53.861100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c6908-a4c6-4387-92ea-c61fd563cdda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:53.862556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6c6908-a4c6-4387-92ea-c61fd563cdda None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:53.877994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 300/598] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:53.878953 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c6908-a4c6-4387-92ea-c61fd563cdda None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:53.993605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 299/599] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:30:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:53.995053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a42b9-9a83-4a3c-9a27-cc6729a3c590 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:54.001083 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6c6908-a4c6-4387-92ea-c61fd563cdda None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:54.039930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deec2d6-1200-46d5-abd4-e928a7fca1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:54.040540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deec2d6-1200-46d5-abd4-e928a7fca1ce None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:54.040938 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deec2d6-1200-46d5-abd4-e928a7fca1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:54.041257 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deec2d6-1200-46d5-abd4-e928a7fca1ce None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:54.042055 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deec2d6-1200-46d5-abd4-e928a7fca1ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:54.042360 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deec2d6-1200-46d5-abd4-e928a7fca1ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:54.043873 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deec2d6-1200-46d5-abd4-e928a7fca1ce None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:54.058688 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 301/600] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:54.059564 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deec2d6-1200-46d5-abd4-e928a7fca1ce None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:54.072891 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:54.105437 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:54.155465 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a42b9-9a83-4a3c-9a27-cc6729a3c590 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:54.201925 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deec2d6-1200-46d5-abd4-e928a7fca1ce None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:54.215174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dff25e-d3b8-4795-81c4-16b6426eeb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:54.216100 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dff25e-d3b8-4795-81c4-16b6426eeb40 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:54.216830 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dff25e-d3b8-4795-81c4-16b6426eeb40 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:54.217158 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dff25e-d3b8-4795-81c4-16b6426eeb40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:54.228537 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dff25e-d3b8-4795-81c4-16b6426eeb40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:54.230951 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dff25e-d3b8-4795-81c4-16b6426eeb40 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:54.248670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-38dff25e-d3b8-4795-81c4-16b6426eeb40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:30:54.248670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:54.248670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:54.248670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:54.248670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:54.248670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:54.248670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:54.248670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:54.248670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:54.248670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:54.248670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:54.248670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:54.248670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:30:54.248670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:30:54.248670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:30:54.249751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:30:54.249751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:54.249751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:54.249751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:54.249751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:54.249751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:54.249751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:54.249751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:54.249751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:54.249751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:54.249751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:54.249751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:54.249751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:54.249751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:54.249751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:54.249751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:54.249751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:54.250715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:30:54.250715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:30:54.250715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:30:54.250715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:30:54.250715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:30:54.250715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:30:54.250715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:30:54.250715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:54.255001 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 300/601] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:30:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:54.256076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dff25e-d3b8-4795-81c4-16b6426eeb40 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:54.256559 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b61ad-68d3-4d2c-b049-88029d553f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:54.257872 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b61ad-68d3-4d2c-b049-88029d553f5a None admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:54.258366 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b61ad-68d3-4d2c-b049-88029d553f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:54.258841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b61ad-68d3-4d2c-b049-88029d553f5a None admin] PATH_INFO: `/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/1347c0bd4ada4139ad194d423958be6e/roles/11e7404599ae49afb5edd22dabe1a5b2` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:54.280003 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b61ad-68d3-4d2c-b049-88029d553f5a None admin] RBAC: Authorizing `identity:create_grant(project_id=a47d6b6971c3452e9799e8146856bdc5, user_id=1347c0bd4ada4139ad194d423958be6e, role_id=11e7404599ae49afb5edd22dabe1a5b2)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:54.282423 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b61ad-68d3-4d2c-b049-88029d553f5a None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:54.375676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 302/602] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:30:54 2022] PUT /identity/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/1347c0bd4ada4139ad194d423958be6e/roles/11e7404599ae49afb5edd22dabe1a5b2 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:30:54.377719 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b61ad-68d3-4d2c-b049-88029d553f5a None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:54.380831 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dff25e-d3b8-4795-81c4-16b6426eeb40 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:54.416124 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:54.453075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc87f65-4849-4004-8bb1-924cf3700b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:54.453953 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc87f65-4849-4004-8bb1-924cf3700b0f None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:54.454240 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc87f65-4849-4004-8bb1-924cf3700b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:54.454562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc87f65-4849-4004-8bb1-924cf3700b0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:54.477991 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc87f65-4849-4004-8bb1-924cf3700b0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:54.479769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc87f65-4849-4004-8bb1-924cf3700b0f None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:54.484449 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:54.513918 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-5dc87f65-4849-4004-8bb1-924cf3700b0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:30:54.513918 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:54.513918 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:54.513918 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:54.513918 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:54.513918 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:54.513918 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:54.513918 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:54.513918 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:54.513918 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:54.513918 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:54.513918 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:54.513918 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:30:54.513918 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:30:54.513918 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:30:54.515320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:30:54.515320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:54.515320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:54.515320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:54.515320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:54.515320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:54.515320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:54.515320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:54.515320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:54.515320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:54.515320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:54.515320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:54.515320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:54.515320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:54.515320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:54.515320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:54.515320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:54.516650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:30:54.516650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:30:54.516650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:30:54.516650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:30:54.516650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:30:54.516650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:30:54.516650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:30:54.516650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:54.516650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 301/603] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:30:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:54.522889 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc87f65-4849-4004-8bb1-924cf3700b0f None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:54.566608 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b61ad-68d3-4d2c-b049-88029d553f5a None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:54.610951 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5203ea7-6188-4352-b3dc-a50982e4fdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:54.612056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5203ea7-6188-4352-b3dc-a50982e4fdce None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:54.612560 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5203ea7-6188-4352-b3dc-a50982e4fdce None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:54.613084 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5203ea7-6188-4352-b3dc-a50982e4fdce None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:54.614755 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5203ea7-6188-4352-b3dc-a50982e4fdce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:54.615324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5203ea7-6188-4352-b3dc-a50982e4fdce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:54.618467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5203ea7-6188-4352-b3dc-a50982e4fdce None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:54.636237 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 303/604] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:30:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:54.644872 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5203ea7-6188-4352-b3dc-a50982e4fdce None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:54.761120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc87f65-4849-4004-8bb1-924cf3700b0f None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:54.821507 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2ed538-d9bf-4a86-89d7-c08efc87060c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:54.822607 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ed538-d9bf-4a86-89d7-c08efc87060c None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:54.823133 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ed538-d9bf-4a86-89d7-c08efc87060c None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:54.823628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2ed538-d9bf-4a86-89d7-c08efc87060c None admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:54.824688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ed538-d9bf-4a86-89d7-c08efc87060c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:54.824829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5203ea7-6188-4352-b3dc-a50982e4fdce None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:54.825378 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ed538-d9bf-4a86-89d7-c08efc87060c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:54.827251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2ed538-d9bf-4a86-89d7-c08efc87060c None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:54.841281 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 302/605] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:30:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:54.842976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6d520-ac58-4a00-afa5-4ad65718efd9 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:54.843492 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6d520-ac58-4a00-afa5-4ad65718efd9 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:54.843990 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd6d520-ac58-4a00-afa5-4ad65718efd9 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:54.845969 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 303/606] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:54.846892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6d520-ac58-4a00-afa5-4ad65718efd9 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:54.852793 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:54.880653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21854bdd-70e6-4fad-afd6-05b93b2d8c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:54.882197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21854bdd-70e6-4fad-afd6-05b93b2d8c4c None admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:54.882869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21854bdd-70e6-4fad-afd6-05b93b2d8c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:54.883216 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21854bdd-70e6-4fad-afd6-05b93b2d8c4c None admin] PATH_INFO: `/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/dd34f15400c4476bbb2161dbe1b8b2b3/roles/11e7404599ae49afb5edd22dabe1a5b2` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:54.906453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21854bdd-70e6-4fad-afd6-05b93b2d8c4c None admin] RBAC: Authorizing `identity:create_grant(project_id=a47d6b6971c3452e9799e8146856bdc5, user_id=dd34f15400c4476bbb2161dbe1b8b2b3, role_id=11e7404599ae49afb5edd22dabe1a5b2)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:54.908693 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21854bdd-70e6-4fad-afd6-05b93b2d8c4c None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:54.910340 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:54.976635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd6d520-ac58-4a00-afa5-4ad65718efd9 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:55.019181 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 304/607] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:30:54 2022] PUT /identity/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/dd34f15400c4476bbb2161dbe1b8b2b3/roles/11e7404599ae49afb5edd22dabe1a5b2 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:30:55.021317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af2351-ee92-4396-b7c7-e0d6400b20d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:55.022415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af2351-ee92-4396-b7c7-e0d6400b20d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:55.022814 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af2351-ee92-4396-b7c7-e0d6400b20d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:55.023106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af2351-ee92-4396-b7c7-e0d6400b20d8 None admin] PATH_INFO: `/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/7b7e90ca75d643cbb0e6d6791fc30b45/roles/11e7404599ae49afb5edd22dabe1a5b2` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:55.023791 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f7c14-05df-41d6-be24-57bdc18d76af None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:55.024288 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f7c14-05df-41d6-be24-57bdc18d76af None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:55.024774 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f7c14-05df-41d6-be24-57bdc18d76af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:55.064629 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-948f7c14-05df-41d6-be24-57bdc18d76af None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:55.065923 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af2351-ee92-4396-b7c7-e0d6400b20d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=a47d6b6971c3452e9799e8146856bdc5, user_id=7b7e90ca75d643cbb0e6d6791fc30b45, role_id=11e7404599ae49afb5edd22dabe1a5b2)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:55.067753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af2351-ee92-4396-b7c7-e0d6400b20d8 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:55.070614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-948f7c14-05df-41d6-be24-57bdc18d76af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:55.164801 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 304/608] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:30:54 2022] PUT /identity/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/7b7e90ca75d643cbb0e6d6791fc30b45/roles/11e7404599ae49afb5edd22dabe1a5b2 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:30:55.199972 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 305/609] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:55.246632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af2351-ee92-4396-b7c7-e0d6400b20d8 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:55.376982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:55.385315 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af2351-ee92-4396-b7c7-e0d6400b20d8 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:55.427764 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ba1b7-ea76-46bd-9da9-931dca27afc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:55.428752 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ba1b7-ea76-46bd-9da9-931dca27afc3 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:55.429101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ba1b7-ea76-46bd-9da9-931dca27afc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:55.429415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ba1b7-ea76-46bd-9da9-931dca27afc3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:55.445268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ba1b7-ea76-46bd-9da9-931dca27afc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:55.453505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-dd5ba1b7-ea76-46bd-9da9-931dca27afc3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:55.453505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:55.453505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:55.453505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:55.453505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:55.453505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:55.453505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:55.453505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:55.453505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:55.453505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:55.453505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:55.453505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:55.453505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:55.453505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:55.453505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:55.453505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:55.454854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:55.454854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:55.454854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:55.454854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:55.454854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:55.454854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:55.454854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:55.454854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:55.454854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:55.454854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:55.454854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:55.454854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:55.454854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:55.454854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:55.454854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:55.454854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:55.456339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:55.456339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:55.456339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:55.456339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:55.456339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:55.456339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:55.456339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:55.456339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:55.456339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:55.456339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:55.456339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:55.456339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:55.456339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 305/610] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:30:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:55.466382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f7c14-05df-41d6-be24-57bdc18d76af None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:55.578749 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:55.584986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f7c14-05df-41d6-be24-57bdc18d76af None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:55.628272 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7ec75-29c4-4fd8-8b7b-d415385f797c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:55.629025 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7ec75-29c4-4fd8-8b7b-d415385f797c None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:55.629316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7ec75-29c4-4fd8-8b7b-d415385f797c None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:55.629597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7ec75-29c4-4fd8-8b7b-d415385f797c None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:55.630427 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7ec75-29c4-4fd8-8b7b-d415385f797c None admin] RBAC: Adding query filter params (name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:55.630735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7ec75-29c4-4fd8-8b7b-d415385f797c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:55.632879 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7ec75-29c4-4fd8-8b7b-d415385f797c None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:55.649868 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 306/611] 10.222.0.13 () {56 vars in 1103 bytes} [Fri May 27 19:30:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:55.655735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ba1b7-ea76-46bd-9da9-931dca27afc3 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:55.902141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ba1b7-ea76-46bd-9da9-931dca27afc3 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:55.972810 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5ec88-362e-4a49-b1c1-dd452d96774d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:55.974191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5ec88-362e-4a49-b1c1-dd452d96774d None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:55.974687 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5ec88-362e-4a49-b1c1-dd452d96774d None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:55.975673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5ec88-362e-4a49-b1c1-dd452d96774d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:55.993775 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b5ec88-362e-4a49-b1c1-dd452d96774d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:55.995441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b5ec88-362e-4a49-b1c1-dd452d96774d None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:56.025708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-50b5ec88-362e-4a49-b1c1-dd452d96774d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:56.025708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:56.025708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:56.025708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:56.025708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:56.025708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:56.025708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:56.025708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:56.025708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:56.025708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:56.025708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:56.025708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:56.025708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:56.025708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:56.025708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:56.027290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:56.027290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:56.027290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:56.027290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:56.027290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:56.027290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:56.027290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:56.027290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:56.027290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:56.027290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:56.027290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:56.027290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:56.027290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:56.027290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:56.027290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:56.027290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:56.027290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:56.030647 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:56.030647 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:56.030647 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:56.030647 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:56.030647 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 306/612] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:56.033646 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7ec75-29c4-4fd8-8b7b-d415385f797c None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:56.257956 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee38414-c7f8-4eae-bc0f-ec091a8277d0 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:56.258914 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee38414-c7f8-4eae-bc0f-ec091a8277d0 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:56.259647 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee38414-c7f8-4eae-bc0f-ec091a8277d0 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:56.263877 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 307/613] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 19:30:56.272125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcf877-98b9-4b0f-9a51-cd30b59c18f1 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:56.273002 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcf877-98b9-4b0f-9a51-cd30b59c18f1 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:56.273724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcf877-98b9-4b0f-9a51-cd30b59c18f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:56.281005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7ec75-29c4-4fd8-8b7b-d415385f797c None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:56.301171 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:56.314638 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-aadcf877-98b9-4b0f-9a51-cd30b59c18f1 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:56.315846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-aadcf877-98b9-4b0f-9a51-cd30b59c18f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:56.348473 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535814c-d011-48b6-b142-320ee92be500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:56.349171 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535814c-d011-48b6-b142-320ee92be500 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:56.349462 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535814c-d011-48b6-b142-320ee92be500 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:56.350276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2535814c-d011-48b6-b142-320ee92be500 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:56.351285 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535814c-d011-48b6-b142-320ee92be500 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:56.351597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535814c-d011-48b6-b142-320ee92be500 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:56.353970 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2535814c-d011-48b6-b142-320ee92be500 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:56.376786 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 307/614] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:56.383341 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535814c-d011-48b6-b142-320ee92be500 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:56.425321 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:56.465044 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 308/615] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:56.509790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcf877-98b9-4b0f-9a51-cd30b59c18f1 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:56.571501 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2535814c-d011-48b6-b142-320ee92be500 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:56.667254 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded03e2-d6fe-4abb-b735-2d6f43be97a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:56.668391 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded03e2-d6fe-4abb-b735-2d6f43be97a9 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:56.668828 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded03e2-d6fe-4abb-b735-2d6f43be97a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:56.669273 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded03e2-d6fe-4abb-b735-2d6f43be97a9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:56.682380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded03e2-d6fe-4abb-b735-2d6f43be97a9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:56.684180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded03e2-d6fe-4abb-b735-2d6f43be97a9 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:56.719984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-9ded03e2-d6fe-4abb-b735-2d6f43be97a9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 19:30:56.719984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:56.719984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:56.719984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:56.719984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:56.719984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:56.719984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:56.719984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:56.719984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:56.719984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:56.719984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:56.719984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:56.719984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:30:56.719984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:30:56.719984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:30:56.719984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:30:56.721087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:56.721087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:56.721087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:30:56.721087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:56.721087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:30:56.721087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:56.721087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:56.721087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:56.721087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:56.721087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:56.721087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:56.721087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:56.721087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:56.721087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:56.721087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:56.721087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:56.721087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:56.721087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:56.722028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:56.722028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:56.722028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:30:56.722028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:30:56.722028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:30:56.722028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:30:56.722028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:30:56.722028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:30:56.722028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 19:30:56.722028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:56.726458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 308/616] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:30:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:56.736976 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded03e2-d6fe-4abb-b735-2d6f43be97a9 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:56.737611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcf877-98b9-4b0f-9a51-cd30b59c18f1 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:56.801572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274cc5c7-9f30-42dd-a78f-2b48f1e21336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:56.802322 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274cc5c7-9f30-42dd-a78f-2b48f1e21336 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:56.802623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274cc5c7-9f30-42dd-a78f-2b48f1e21336 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:56.802919 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274cc5c7-9f30-42dd-a78f-2b48f1e21336 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:56.806104 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:56.814821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274cc5c7-9f30-42dd-a78f-2b48f1e21336 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:56.824554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-274cc5c7-9f30-42dd-a78f-2b48f1e21336 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:56.824554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:56.824554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:56.824554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:56.824554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:56.824554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:56.824554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:56.824554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:56.824554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:56.824554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:56.824554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:56.824554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:56.824554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:56.824554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:56.824554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:56.824554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:56.825550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:56.825550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:56.825550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:56.825550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:56.825550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:56.825550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:56.825550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:56.825550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:56.825550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:56.825550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:56.825550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:56.825550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:56.825550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:56.825550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:56.825550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:56.825550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:56.826458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:56.826458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:56.826458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:56.826458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:56.826458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:56.826458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:56.826458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:56.826458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:56.826458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:56.826458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:56.826458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:56.826458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:56.826458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 309/617] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:30:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:56.840981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274cc5c7-9f30-42dd-a78f-2b48f1e21336 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:56.906590 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded03e2-d6fe-4abb-b735-2d6f43be97a9 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:56.909260 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:56.975126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85707dd0-a93f-4d79-a4c8-765207107ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:56.980973 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85707dd0-a93f-4d79-a4c8-765207107ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:56.980973 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85707dd0-a93f-4d79-a4c8-765207107ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:56.980973 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85707dd0-a93f-4d79-a4c8-765207107ea5 None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:56.980973 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85707dd0-a93f-4d79-a4c8-765207107ea5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:56.980973 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85707dd0-a93f-4d79-a4c8-765207107ea5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:56.983812 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85707dd0-a93f-4d79-a4c8-765207107ea5 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:57.031965 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 309/618] 10.222.0.13 () {56 vars in 1135 bytes} [Fri May 27 19:30:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:57.036159 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85707dd0-a93f-4d79-a4c8-765207107ea5 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:57.065055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274cc5c7-9f30-42dd-a78f-2b48f1e21336 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:57.110043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b461803-6ba5-4f49-9b18-441afb59b629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:57.110738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b461803-6ba5-4f49-9b18-441afb59b629 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:57.111018 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b461803-6ba5-4f49-9b18-441afb59b629 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:57.111292 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b461803-6ba5-4f49-9b18-441afb59b629 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:57.112062 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b461803-6ba5-4f49-9b18-441afb59b629 None admin] RBAC: Adding query filter params (name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:57.112370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b461803-6ba5-4f49-9b18-441afb59b629 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:57.118129 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b461803-6ba5-4f49-9b18-441afb59b629 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:57.147676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 310/619] 10.222.0.13 () {56 vars in 1103 bytes} [Fri May 27 19:30:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:57.154600 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b461803-6ba5-4f49-9b18-441afb59b629 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:57.221271 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85707dd0-a93f-4d79-a4c8-765207107ea5 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:57.299245 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c105da20-45ef-40ef-9268-b2f189cdad05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:57.302150 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105da20-45ef-40ef-9268-b2f189cdad05 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:57.304990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105da20-45ef-40ef-9268-b2f189cdad05 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:57.304990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c105da20-45ef-40ef-9268-b2f189cdad05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:57.307205 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:57.324973 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:57.326433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c105da20-45ef-40ef-9268-b2f189cdad05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:57.327947 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c105da20-45ef-40ef-9268-b2f189cdad05 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:57.353620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-c105da20-45ef-40ef-9268-b2f189cdad05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:57.353620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:57.353620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:57.353620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:57.353620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:57.353620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:57.353620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:57.353620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:57.353620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:57.353620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:57.353620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:57.353620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:57.353620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:57.353620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:57.353620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:57.354741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:57.354741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:57.354741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:57.354741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:57.354741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:57.354741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:57.354741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:57.354741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:57.354741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:57.354741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:57.354741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:57.354741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:57.354741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:57.354741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:57.354741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:57.354741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:57.354741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:57.355655 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:57.355655 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:57.355655 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:57.355655 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:57.363102 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 310/620] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:57.363102 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2f5cd-6512-4fa3-ab84-648a49f9f579 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:57.363102 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2f5cd-6512-4fa3-ab84-648a49f9f579 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:57.363102 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2f5cd-6512-4fa3-ab84-648a49f9f579 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:57.364442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 311/621] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:57.365937 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2f5cd-6512-4fa3-ab84-648a49f9f579 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:57.377293 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b461803-6ba5-4f49-9b18-441afb59b629 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:57.423891 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb57391-911a-4a0d-8c6c-153504bb42b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:57.425690 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb57391-911a-4a0d-8c6c-153504bb42b2 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:57.426607 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb57391-911a-4a0d-8c6c-153504bb42b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:57.427090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb57391-911a-4a0d-8c6c-153504bb42b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:57.442198 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb57391-911a-4a0d-8c6c-153504bb42b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:57.444762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb57391-911a-4a0d-8c6c-153504bb42b2 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:57.469075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-dcb57391-911a-4a0d-8c6c-153504bb42b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:57.469075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:57.469075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:57.469075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:57.469075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:57.469075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:57.469075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:57.469075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:57.469075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:57.469075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:57.469075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:57.469075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:57.469075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:57.469075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:57.469075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:57.470501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:57.470501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:57.470501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:57.470501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:57.470501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:57.470501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:57.470501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:57.470501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:57.470501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:57.470501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:57.470501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:57.470501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:57.470501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:57.470501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:57.470501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:57.470501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:57.470501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:57.471406 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:57.471406 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:57.471406 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:57.471406 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:57.474138 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 311/622] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:57.475693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb5374-9a23-4d7a-9c6e-0bd1eb82b9d9 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:57.476177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb5374-9a23-4d7a-9c6e-0bd1eb82b9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:57.476583 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fb5374-9a23-4d7a-9c6e-0bd1eb82b9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:57.500210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-23fb5374-9a23-4d7a-9c6e-0bd1eb82b9d9 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:57.501954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-23fb5374-9a23-4d7a-9c6e-0bd1eb82b9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:57.502571 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2f5cd-6512-4fa3-ab84-648a49f9f579 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:57.551404 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29089bc5-5693-4b2c-9843-72da36363148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:57.552223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29089bc5-5693-4b2c-9843-72da36363148 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:57.553164 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29089bc5-5693-4b2c-9843-72da36363148 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:57.553611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29089bc5-5693-4b2c-9843-72da36363148 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:57.554587 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29089bc5-5693-4b2c-9843-72da36363148 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:57.554993 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29089bc5-5693-4b2c-9843-72da36363148 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:57.556617 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29089bc5-5693-4b2c-9843-72da36363148 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:57.574347 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 312/623] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:57.574347 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29089bc5-5693-4b2c-9843-72da36363148 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:57.598796 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 312/624] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:57.600173 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb5374-9a23-4d7a-9c6e-0bd1eb82b9d9 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:57.645766 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:57.679246 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:57.733023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29089bc5-5693-4b2c-9843-72da36363148 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:57.770409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fb5374-9a23-4d7a-9c6e-0bd1eb82b9d9 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:57.780486 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2eff29-9884-4158-86c0-680fdd1fb20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:57.781198 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2eff29-9884-4158-86c0-680fdd1fb20c None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:57.781492 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2eff29-9884-4158-86c0-680fdd1fb20c None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:57.781773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2eff29-9884-4158-86c0-680fdd1fb20c None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:57.782512 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2eff29-9884-4158-86c0-680fdd1fb20c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:57.782820 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2eff29-9884-4158-86c0-680fdd1fb20c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:57.784201 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2eff29-9884-4158-86c0-680fdd1fb20c None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:57.802734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 313/625] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:57.802734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2eff29-9884-4158-86c0-680fdd1fb20c None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:57.816845 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554c9f6-73af-4d6e-83a9-99a3c513f1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:57.818635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554c9f6-73af-4d6e-83a9-99a3c513f1fc None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:57.819528 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554c9f6-73af-4d6e-83a9-99a3c513f1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:57.820022 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e554c9f6-73af-4d6e-83a9-99a3c513f1fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:57.828525 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e554c9f6-73af-4d6e-83a9-99a3c513f1fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:57.830434 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e554c9f6-73af-4d6e-83a9-99a3c513f1fc None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:57.846163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-e554c9f6-73af-4d6e-83a9-99a3c513f1fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:30:57.846163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:57.846163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:57.846163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:57.846163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:57.846163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:57.846163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:57.846163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:57.846163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:57.846163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:57.846163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:57.846163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:57.846163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:30:57.846163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:30:57.846163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:30:57.847149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:30:57.847149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:57.847149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:57.847149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:57.847149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:57.847149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:57.847149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:57.847149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:57.847149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:57.847149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:57.847149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:57.847149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:57.847149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:57.847149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:57.847149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:57.847149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:57.847149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:57.848066 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:30:57.848066 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:30:57.848066 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:30:57.848066 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:30:57.848066 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:30:57.848066 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:30:57.848066 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:30:57.848066 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:57.851354 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 313/626] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:30:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:57.852429 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554c9f6-73af-4d6e-83a9-99a3c513f1fc None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:57.932732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2eff29-9884-4158-86c0-680fdd1fb20c None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:57.944617 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:57.960851 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:57.964068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e554c9f6-73af-4d6e-83a9-99a3c513f1fc None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:57.969355 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a18c36-d37f-462e-9157-3121b2adbf3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:57.970161 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a18c36-d37f-462e-9157-3121b2adbf3e None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:57.970446 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a18c36-d37f-462e-9157-3121b2adbf3e None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:57.970724 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a18c36-d37f-462e-9157-3121b2adbf3e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:57.981821 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a18c36-d37f-462e-9157-3121b2adbf3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:57.991065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-72a18c36-d37f-462e-9157-3121b2adbf3e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:57.991065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:57.991065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:57.991065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:57.991065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:57.991065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:57.991065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:57.991065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:57.991065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:57.991065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:57.991065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:57.991065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:57.991065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:57.991065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:57.991065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:57.991065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:57.998940 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:57.998940 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:57.998940 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:57.998940 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:57.998940 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:57.998940 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:57.998940 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:57.998940 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:57.998940 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:57.998940 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:57.998940 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:57.998940 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:57.998940 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:57.998940 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:57.998940 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:57.998940 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:57.999819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:57.999819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:57.999819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:57.999819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:57.999819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:57.999819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:57.999819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:57.999819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:57.999819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:57.999819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:57.999819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:57.999819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:57.999819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 314/627] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:30:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:57.999819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69437a4c-2cea-4b53-af25-217f8ab46181 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:58.000584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69437a4c-2cea-4b53-af25-217f8ab46181 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:58.000584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69437a4c-2cea-4b53-af25-217f8ab46181 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:58.000584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 315/628] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:58.000584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69437a4c-2cea-4b53-af25-217f8ab46181 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:58.003247 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0989eb-41d9-41ce-8f99-1de175a4de4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:58.004055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0989eb-41d9-41ce-8f99-1de175a4de4e None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:58.004344 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0989eb-41d9-41ce-8f99-1de175a4de4e None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:58.004617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0989eb-41d9-41ce-8f99-1de175a4de4e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:58.012724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0989eb-41d9-41ce-8f99-1de175a4de4e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:58.014219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0989eb-41d9-41ce-8f99-1de175a4de4e None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:58.029672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-df0989eb-41d9-41ce-8f99-1de175a4de4e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 19:30:58.029672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:58.029672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:58.029672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:58.029672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:58.029672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:58.029672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:58.029672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:58.029672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:58.029672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:58.029672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:58.029672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:58.029672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:30:58.029672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:30:58.029672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:30:58.029672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:30:58.030880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:58.030880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:58.030880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:30:58.030880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:58.030880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:30:58.030880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:58.030880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:58.030880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:58.030880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:58.030880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:58.030880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:58.030880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:58.030880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:58.030880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:58.030880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:58.030880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:58.030880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:58.030880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:58.031988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:58.031988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:58.031988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:30:58.031988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:30:58.031988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:30:58.031988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:30:58.031988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:30:58.031988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:30:58.031988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 19:30:58.031988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:58.031988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 314/629] 10.222.0.13 () {56 vars in 1093 bytes} [Fri May 27 19:30:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:58.034442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e488ab2-9351-4ee9-9901-38b18f08ab10 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:58.034736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e488ab2-9351-4ee9-9901-38b18f08ab10 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:58.035014 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e488ab2-9351-4ee9-9901-38b18f08ab10 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:58.037548 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 315/630] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:30:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:30:58.038266 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e488ab2-9351-4ee9-9901-38b18f08ab10 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:58.133673 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69437a4c-2cea-4b53-af25-217f8ab46181 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:58.153649 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e488ab2-9351-4ee9-9901-38b18f08ab10 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:58.171714 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac3516-ceb9-48a7-8f38-7517de489727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:58.172370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac3516-ceb9-48a7-8f38-7517de489727 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:58.172655 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac3516-ceb9-48a7-8f38-7517de489727 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:58.173019 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac3516-ceb9-48a7-8f38-7517de489727 None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:58.173858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac3516-ceb9-48a7-8f38-7517de489727 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:58.174171 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac3516-ceb9-48a7-8f38-7517de489727 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:58.175643 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac3516-ceb9-48a7-8f38-7517de489727 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:58.190224 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 316/631] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:30:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:58.191857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a26494-5e6b-4e72-a043-4c5b95aef5d8 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:58.192369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a26494-5e6b-4e72-a043-4c5b95aef5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:58.192789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a26494-5e6b-4e72-a043-4c5b95aef5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:58.197802 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da91689-998d-4685-8097-1173a62fcdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:58.198479 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da91689-998d-4685-8097-1173a62fcdb1 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:58.198765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da91689-998d-4685-8097-1173a62fcdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:58.199049 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da91689-998d-4685-8097-1173a62fcdb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:58.199808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da91689-998d-4685-8097-1173a62fcdb1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:58.200121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da91689-998d-4685-8097-1173a62fcdb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:58.201583 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da91689-998d-4685-8097-1173a62fcdb1 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:58.211402 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 316/632] 10.222.0.13 () {56 vars in 1103 bytes} [Fri May 27 19:30:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:58.211867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-53a26494-5e6b-4e72-a043-4c5b95aef5d8 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:58.212224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da91689-998d-4685-8097-1173a62fcdb1 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:58.213111 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-53a26494-5e6b-4e72-a043-4c5b95aef5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:58.238012 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:58.239431 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:58.289113 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 317/633] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:58.290499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e382c00-7d45-44b0-97f6-b17171cfb25f None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:58.290791 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e382c00-7d45-44b0-97f6-b17171cfb25f None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:58.291066 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e382c00-7d45-44b0-97f6-b17171cfb25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:58.309295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-1e382c00-7d45-44b0-97f6-b17171cfb25f None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:30:58.310483 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-1e382c00-7d45-44b0-97f6-b17171cfb25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:30:58.322499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da91689-998d-4685-8097-1173a62fcdb1 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:58.360176 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef88e7-7c39-4fef-849e-23e936522cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:58.360886 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef88e7-7c39-4fef-849e-23e936522cce None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:58.361204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef88e7-7c39-4fef-849e-23e936522cce None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:58.361497 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef88e7-7c39-4fef-849e-23e936522cce None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:58.362505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef88e7-7c39-4fef-849e-23e936522cce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:58.362821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef88e7-7c39-4fef-849e-23e936522cce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:58.364348 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ef88e7-7c39-4fef-849e-23e936522cce None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:58.384352 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 317/634] 10.222.0.13 () {56 vars in 1133 bytes} [Fri May 27 19:30:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:58.390186 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 318/635] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:30:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:30:58.390355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef88e7-7c39-4fef-849e-23e936522cce None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:58.390437 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e382c00-7d45-44b0-97f6-b17171cfb25f None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:58.502635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:58.509612 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:58.520215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e382c00-7d45-44b0-97f6-b17171cfb25f None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:58.526956 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef88e7-7c39-4fef-849e-23e936522cce None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:58.557747 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d8b92-2af6-4ea0-9643-a715770a757a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:58.558715 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d8b92-2af6-4ea0-9643-a715770a757a None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:58.559004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d8b92-2af6-4ea0-9643-a715770a757a None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:58.559296 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d8b92-2af6-4ea0-9643-a715770a757a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:58.571436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d8b92-2af6-4ea0-9643-a715770a757a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:58.572911 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d8b92-2af6-4ea0-9643-a715770a757a None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:58.573419 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1716579-aedf-4f63-b41a-92e921e07f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:58.574409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1716579-aedf-4f63-b41a-92e921e07f72 None admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:58.574708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1716579-aedf-4f63-b41a-92e921e07f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:58.574962 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1716579-aedf-4f63-b41a-92e921e07f72 None admin] PATH_INFO: `/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/37bc6d0abb454de19e1ab2f46bf4a593/roles/11e7404599ae49afb5edd22dabe1a5b2` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:58.592405 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1716579-aedf-4f63-b41a-92e921e07f72 None admin] RBAC: Authorizing `identity:create_grant(project_id=a47d6b6971c3452e9799e8146856bdc5, user_id=37bc6d0abb454de19e1ab2f46bf4a593, role_id=11e7404599ae49afb5edd22dabe1a5b2)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:58.593893 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1716579-aedf-4f63-b41a-92e921e07f72 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:58.594467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-091d8b92-2af6-4ea0-9643-a715770a757a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:58.594467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:58.594467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:58.594467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:58.594467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:58.594467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:58.594467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:58.594467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:58.594467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:58.594467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:58.594467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:58.594467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:58.594467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:58.594467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:58.594467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:58.595421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:58.595421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:58.595421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:58.595421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:58.595421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:58.595421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:58.595421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:58.595421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:58.595421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:58.595421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:58.595421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:58.595421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:58.595421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:58.595421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:58.595421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:58.595421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:58.595421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:58.596330 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:58.596330 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:58.596330 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:58.596330 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:58.596330 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 319/636] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:58.596615 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d8b92-2af6-4ea0-9643-a715770a757a None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:58.677716 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 318/637] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:30:58 2022] PUT /identity/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/37bc6d0abb454de19e1ab2f46bf4a593/roles/11e7404599ae49afb5edd22dabe1a5b2 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:30:58.679113 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1716579-aedf-4f63-b41a-92e921e07f72 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:58.727587 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d8b92-2af6-4ea0-9643-a715770a757a None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:58.775557 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e71372-4e5a-4804-8e80-2e8b2f3d7417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:58.776386 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e71372-4e5a-4804-8e80-2e8b2f3d7417 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:58.776795 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e71372-4e5a-4804-8e80-2e8b2f3d7417 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:58.777342 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e71372-4e5a-4804-8e80-2e8b2f3d7417 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:58.791827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e71372-4e5a-4804-8e80-2e8b2f3d7417 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:58.800475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-f5e71372-4e5a-4804-8e80-2e8b2f3d7417 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:58.800475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:58.800475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:58.800475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:58.800475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:58.800475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:58.800475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:58.800475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:58.800475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:58.800475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:58.800475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:58.800475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:58.800475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:58.800475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:58.800475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:58.800475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:58.801864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:58.801864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:58.801864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:58.801864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:58.801864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:58.801864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:58.801864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:58.801864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:58.801864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:58.801864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:58.801864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:58.801864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:58.801864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:58.801864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:58.801864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:58.801864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:58.803567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:58.803567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:58.803567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:58.803567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:58.803567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:58.803567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:58.803567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:58.803567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:58.803567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:58.803567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:58.803567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:58.803567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:58.804238 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 320/638] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:30:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:58.805169 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e71372-4e5a-4804-8e80-2e8b2f3d7417 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:58.808646 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:58.824111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1716579-aedf-4f63-b41a-92e921e07f72 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:58.873351 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:58.882489 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07fad9-ca4e-41a2-85a1-e45af7d918d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:58.884032 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07fad9-ca4e-41a2-85a1-e45af7d918d6 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:58.884032 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07fad9-ca4e-41a2-85a1-e45af7d918d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:58.884032 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07fad9-ca4e-41a2-85a1-e45af7d918d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:58.941842 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e71372-4e5a-4804-8e80-2e8b2f3d7417 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:58.976333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b07fad9-ca4e-41a2-85a1-e45af7d918d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:58.979216 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b07fad9-ca4e-41a2-85a1-e45af7d918d6 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:58.988593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eaa643-4a26-41ed-be5d-2a50eaca9fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:58.990642 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eaa643-4a26-41ed-be5d-2a50eaca9fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:58.991275 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eaa643-4a26-41ed-be5d-2a50eaca9fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:58.991866 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eaa643-4a26-41ed-be5d-2a50eaca9fb7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:59.007447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eaa643-4a26-41ed-be5d-2a50eaca9fb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:30:59.015680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-f9eaa643-4a26-41ed-be5d-2a50eaca9fb7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:59.015680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:59.015680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:59.015680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:59.015680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:59.015680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:59.015680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:59.015680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:59.015680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:59.015680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:59.015680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:59.015680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:59.015680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:30:59.015680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:30:59.015680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:30:59.015680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:30:59.016784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:30:59.016784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:30:59.016784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:30:59.016784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:30:59.016784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:59.016784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:59.016784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:59.016784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:59.016784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:59.016784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:59.016784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:59.016784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:59.016784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:59.016784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:59.016784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:59.016784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:59.017767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:59.017767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:59.017767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:59.017767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:59.017767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:30:59.017767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:30:59.017767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:30:59.017767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:30:59.017767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:30:59.017767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:30:59.017767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 19:30:59.017767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:59.017767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 321/639] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:30:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:59.018557 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eaa643-4a26-41ed-be5d-2a50eaca9fb7 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:59.108439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 319/640] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:30:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:30:59.109610 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07fad9-ca4e-41a2-85a1-e45af7d918d6 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:59.118538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:59.147818 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:59.169346 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eaa643-4a26-41ed-be5d-2a50eaca9fb7 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:59.208181 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50adc5fe-f208-4867-8b3f-bd4e9218b6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:59.209003 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50adc5fe-f208-4867-8b3f-bd4e9218b6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:59.209514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50adc5fe-f208-4867-8b3f-bd4e9218b6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:59.209789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50adc5fe-f208-4867-8b3f-bd4e9218b6f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:59.210517 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50adc5fe-f208-4867-8b3f-bd4e9218b6f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:59.210824 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50adc5fe-f208-4867-8b3f-bd4e9218b6f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:59.212206 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50adc5fe-f208-4867-8b3f-bd4e9218b6f8 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:59.229362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 322/641] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:59.229362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50adc5fe-f208-4867-8b3f-bd4e9218b6f8 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:59.229648 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07fad9-ca4e-41a2-85a1-e45af7d918d6 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:59.273656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300d4dc-19fa-4f8a-b4b2-6cad71c7d878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:59.274343 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300d4dc-19fa-4f8a-b4b2-6cad71c7d878 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:59.274632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300d4dc-19fa-4f8a-b4b2-6cad71c7d878 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:59.274911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300d4dc-19fa-4f8a-b4b2-6cad71c7d878 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:59.275730 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300d4dc-19fa-4f8a-b4b2-6cad71c7d878 None admin] RBAC: Adding query filter params (name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:59.276042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300d4dc-19fa-4f8a-b4b2-6cad71c7d878 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:59.277554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300d4dc-19fa-4f8a-b4b2-6cad71c7d878 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:59.291356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 320/642] 10.222.0.13 () {56 vars in 1103 bytes} [Fri May 27 19:30:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:59.291356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300d4dc-19fa-4f8a-b4b2-6cad71c7d878 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:59.354901 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50adc5fe-f208-4867-8b3f-bd4e9218b6f8 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:59.381152 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:59.393125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70305ff7-b9e6-4528-9994-622899bacb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:59.401037 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70305ff7-b9e6-4528-9994-622899bacb59 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:59.401037 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70305ff7-b9e6-4528-9994-622899bacb59 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:59.401037 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70305ff7-b9e6-4528-9994-622899bacb59 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:59.401037 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70305ff7-b9e6-4528-9994-622899bacb59 None admin] RBAC: Adding query filter params (name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:59.401037 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70305ff7-b9e6-4528-9994-622899bacb59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:59.401037 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70305ff7-b9e6-4528-9994-622899bacb59 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:59.404036 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:59.407200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 323/643] 10.222.0.13 () {56 vars in 1103 bytes} [Fri May 27 19:30:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:59.410028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70305ff7-b9e6-4528-9994-622899bacb59 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:59.414162 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300d4dc-19fa-4f8a-b4b2-6cad71c7d878 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:59.455911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442b974-47c6-49ca-9bd4-353ae8c9163f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:59.456724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442b974-47c6-49ca-9bd4-353ae8c9163f None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:59.457028 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442b974-47c6-49ca-9bd4-353ae8c9163f None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:59.457308 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9442b974-47c6-49ca-9bd4-353ae8c9163f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:59.468698 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9442b974-47c6-49ca-9bd4-353ae8c9163f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:59.470163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9442b974-47c6-49ca-9bd4-353ae8c9163f None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:59.490110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-9442b974-47c6-49ca-9bd4-353ae8c9163f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:59.490110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:59.490110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:59.490110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:59.490110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:59.490110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:59.490110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:59.490110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:59.490110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:59.490110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:59.490110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:59.490110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:59.490110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:59.490110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:59.490110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:59.491408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:59.491408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:59.491408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:59.491408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:59.491408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:59.491408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:59.491408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:59.491408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:59.491408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:59.491408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:59.491408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:59.491408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:59.491408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:59.491408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:59.491408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:59.491408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:59.491408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:59.492512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:59.492512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:59.492512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:59.492512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:59.492512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 321/644] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:59.492846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442b974-47c6-49ca-9bd4-353ae8c9163f None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:59.540267 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70305ff7-b9e6-4528-9994-622899bacb59 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:59.592744 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3dfd90-d218-4ea8-9ca4-ed4ff03ad17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:59.594100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3dfd90-d218-4ea8-9ca4-ed4ff03ad17d None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:59.594578 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3dfd90-d218-4ea8-9ca4-ed4ff03ad17d None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:59.595033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3dfd90-d218-4ea8-9ca4-ed4ff03ad17d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:59.608543 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3dfd90-d218-4ea8-9ca4-ed4ff03ad17d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:59.611281 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3dfd90-d218-4ea8-9ca4-ed4ff03ad17d None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:59.613296 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9442b974-47c6-49ca-9bd4-353ae8c9163f None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:59.637178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-2a3dfd90-d218-4ea8-9ca4-ed4ff03ad17d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 19:30:59.637178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:59.637178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:59.637178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:59.637178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:59.637178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:59.637178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:59.637178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:59.637178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:59.637178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:59.637178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:59.637178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:59.637178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:30:59.637178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:30:59.637178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:30:59.637178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:30:59.638416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:59.638416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:59.638416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:30:59.638416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:59.638416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:30:59.638416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:30:59.638416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:59.638416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:59.638416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:59.638416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:59.638416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:59.638416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:59.638416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:59.638416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:59.638416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:59.638416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:59.638416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:59.638416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:59.639622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:59.639622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:59.639622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:30:59.639622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:30:59.639622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:30:59.639622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:30:59.639622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:30:59.639622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:30:59.639622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 19:30:59.639622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:59.639622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 324/645] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:30:59 2022] GET /identity/v3/users/glance => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:59.640530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3dfd90-d218-4ea8-9ca4-ed4ff03ad17d None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:59.659788 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6db93d-48eb-4ae3-ace0-1e96dc1e102e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:59.660736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6db93d-48eb-4ae3-ace0-1e96dc1e102e None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:59.661150 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6db93d-48eb-4ae3-ace0-1e96dc1e102e None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:59.661524 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6db93d-48eb-4ae3-ace0-1e96dc1e102e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:59.676493 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6db93d-48eb-4ae3-ace0-1e96dc1e102e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:59.678680 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6db93d-48eb-4ae3-ace0-1e96dc1e102e None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:59.681831 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:59.699651 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:59.700692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-8f6db93d-48eb-4ae3-ace0-1e96dc1e102e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:59.700692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:59.700692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:59.700692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:59.700692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:59.700692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:59.700692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:59.700692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:59.700692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:59.700692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:59.700692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:59.700692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:59.700692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:59.700692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:59.700692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:59.701799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:59.701799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:59.701799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:59.701799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:59.701799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:59.701799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:59.701799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:59.701799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:59.701799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:30:59.701799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:59.701799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:59.701799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:59.701799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:59.701799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:59.701799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:59.701799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:59.701799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:59.702998 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:59.702998 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:59.702998 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:59.702998 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:30:59.707115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 322/646] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:59.707976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6db93d-48eb-4ae3-ace0-1e96dc1e102e None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:59.761247 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3dfd90-d218-4ea8-9ca4-ed4ff03ad17d None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:59.799864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e6096-c68d-45a1-97c5-81873fb8b9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:59.800664 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e6096-c68d-45a1-97c5-81873fb8b9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:59.800972 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e6096-c68d-45a1-97c5-81873fb8b9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:59.801256 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e6096-c68d-45a1-97c5-81873fb8b9b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:59.815108 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e6096-c68d-45a1-97c5-81873fb8b9b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:59.816546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e6096-c68d-45a1-97c5-81873fb8b9b3 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:59.821570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6db93d-48eb-4ae3-ace0-1e96dc1e102e None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:30:59.835645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-547e6096-c68d-45a1-97c5-81873fb8b9b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:59.835645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:30:59.835645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:30:59.835645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:30:59.835645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:30:59.835645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:30:59.835645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:30:59.835645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:30:59.835645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:30:59.835645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:30:59.835645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:30:59.835645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:30:59.835645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:30:59.835645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:30:59.835645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:30:59.836797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:30:59.836797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:30:59.836797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:30:59.836797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:30:59.836797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:30:59.836797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:30:59.836797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:30:59.836797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:30:59.836797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:30:59.836797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:30:59.836797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:30:59.836797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:30:59.836797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:30:59.836797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:30:59.836797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:30:59.836797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:30:59.836797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:30:59.837756 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:30:59.837756 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:30:59.837756 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:30:59.837756 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:30:59.837756 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 325/647] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:30:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:30:59.837756 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e6096-c68d-45a1-97c5-81873fb8b9b3 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:59.867874 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252922ee-60d8-48c1-b95e-9171996ed656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:30:59.868814 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252922ee-60d8-48c1-b95e-9171996ed656 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:30:59.869120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252922ee-60d8-48c1-b95e-9171996ed656 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:30:59.869398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252922ee-60d8-48c1-b95e-9171996ed656 None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:30:59.870294 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252922ee-60d8-48c1-b95e-9171996ed656 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:30:59.870614 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252922ee-60d8-48c1-b95e-9171996ed656 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:30:59.873353 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252922ee-60d8-48c1-b95e-9171996ed656 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:30:59.896810 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 323/648] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:30:59.896810 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252922ee-60d8-48c1-b95e-9171996ed656 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:30:59.950328 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:59.961340 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:30:59.968500 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e6096-c68d-45a1-97c5-81873fb8b9b3 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:00.010227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c90db8-b7b7-4d6e-94c2-132d798a4658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:00.010484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252922ee-60d8-48c1-b95e-9171996ed656 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:00.010955 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c90db8-b7b7-4d6e-94c2-132d798a4658 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:00.011391 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c90db8-b7b7-4d6e-94c2-132d798a4658 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:00.011727 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c90db8-b7b7-4d6e-94c2-132d798a4658 None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:00.012807 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c90db8-b7b7-4d6e-94c2-132d798a4658 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:00.013096 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c90db8-b7b7-4d6e-94c2-132d798a4658 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:00.014767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c90db8-b7b7-4d6e-94c2-132d798a4658 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:00.034175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 326/649] 10.222.0.13 () {56 vars in 1137 bytes} [Fri May 27 19:30:59 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:00.035163 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c90db8-b7b7-4d6e-94c2-132d798a4658 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:00.064025 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f2ed2-9c94-4581-9d35-4eafcf0c6a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:00.064800 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f2ed2-9c94-4581-9d35-4eafcf0c6a0f None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:00.065211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f2ed2-9c94-4581-9d35-4eafcf0c6a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:00.065553 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11f2ed2-9c94-4581-9d35-4eafcf0c6a0f None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:00.066360 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f2ed2-9c94-4581-9d35-4eafcf0c6a0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:00.066828 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f2ed2-9c94-4581-9d35-4eafcf0c6a0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:00.068662 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11f2ed2-9c94-4581-9d35-4eafcf0c6a0f None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:00.082839 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 324/650] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:00.084328 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f2ed2-9c94-4581-9d35-4eafcf0c6a0f None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:00.151894 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c90db8-b7b7-4d6e-94c2-132d798a4658 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:00.189911 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769a68a-afbe-4fe4-bb8f-bae7d6216342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:00.190588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769a68a-afbe-4fe4-bb8f-bae7d6216342 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:00.190874 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769a68a-afbe-4fe4-bb8f-bae7d6216342 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:00.191150 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6769a68a-afbe-4fe4-bb8f-bae7d6216342 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:00.191880 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6769a68a-afbe-4fe4-bb8f-bae7d6216342 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:00.192185 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6769a68a-afbe-4fe4-bb8f-bae7d6216342 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:00.193865 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6769a68a-afbe-4fe4-bb8f-bae7d6216342 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:00.206723 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 327/651] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:00.207928 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769a68a-afbe-4fe4-bb8f-bae7d6216342 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:00.208077 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11f2ed2-9c94-4581-9d35-4eafcf0c6a0f None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:00.227930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:00.229651 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:00.247775 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d5839a-15f7-4088-a03d-450a3373ecd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:00.248681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5839a-15f7-4088-a03d-450a3373ecd4 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:00.249064 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5839a-15f7-4088-a03d-450a3373ecd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:00.249433 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d5839a-15f7-4088-a03d-450a3373ecd4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:00.256708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d5839a-15f7-4088-a03d-450a3373ecd4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:00.258279 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d5839a-15f7-4088-a03d-450a3373ecd4 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:00.273841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-d1d5839a-15f7-4088-a03d-450a3373ecd4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:00.273841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:00.273841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:00.273841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:00.273841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:00.273841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:00.273841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:00.273841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:00.273841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:00.273841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:00.273841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:00.273841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:00.273841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:31:00.273841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:31:00.273841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:31:00.274769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:31:00.274769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:00.274769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:00.274769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:00.274769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:00.274769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:00.274769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:00.274769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:00.274769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:00.274769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:00.274769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:00.274769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:00.274769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:00.274769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:00.274769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:00.274769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:00.274769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:00.275671 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:31:00.275671 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:31:00.275671 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:31:00.275671 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:31:00.275671 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:31:00.275671 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:31:00.275671 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:00.275671 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:00.275671 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 325/652] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:00.276159 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d5839a-15f7-4088-a03d-450a3373ecd4 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:00.328682 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6769a68a-afbe-4fe4-bb8f-bae7d6216342 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:00.366446 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d1ac8-5b2f-44de-8d0b-7929024b9147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:00.367226 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d1ac8-5b2f-44de-8d0b-7929024b9147 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:00.367590 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d1ac8-5b2f-44de-8d0b-7929024b9147 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:00.367937 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d1ac8-5b2f-44de-8d0b-7929024b9147 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:00.386645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d5839a-15f7-4088-a03d-450a3373ecd4 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:00.424794 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd91dad-0281-4c8b-a626-0dbbcd3679ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:00.425653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd91dad-0281-4c8b-a626-0dbbcd3679ed None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:00.425993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd91dad-0281-4c8b-a626-0dbbcd3679ed None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:00.426278 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd91dad-0281-4c8b-a626-0dbbcd3679ed None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:00.435476 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd91dad-0281-4c8b-a626-0dbbcd3679ed None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:00.436940 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd91dad-0281-4c8b-a626-0dbbcd3679ed None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:00.445157 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d1ac8-5b2f-44de-8d0b-7929024b9147 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:00.446629 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d1ac8-5b2f-44de-8d0b-7929024b9147 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:00.452370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-5bd91dad-0281-4c8b-a626-0dbbcd3679ed None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 19:31:00.452370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:00.452370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:00.452370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:00.452370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:00.452370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:00.452370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:00.452370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:00.452370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:00.452370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:00.452370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:00.452370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:00.452370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:31:00.452370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:31:00.452370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:31:00.452370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:31:00.453430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:00.453430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:00.453430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:31:00.453430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:00.453430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:31:00.453430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:00.453430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:00.453430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:00.453430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:00.453430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:00.453430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:00.453430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:00.453430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:00.453430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:00.453430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:00.453430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:00.453430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:00.453430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:00.454391 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:00.454391 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:00.454391 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:31:00.454391 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:31:00.454391 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:31:00.454391 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:31:00.454391 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:31:00.454391 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:31:00.454391 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 19:31:00.454391 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:00.454391 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 326/653] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:31:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:00.454997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd91dad-0281-4c8b-a626-0dbbcd3679ed None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:00.466715 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:00.498962 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:00.570774 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 328/654] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:31:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:31:00.575936 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d1ac8-5b2f-44de-8d0b-7929024b9147 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:00.584123 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd91dad-0281-4c8b-a626-0dbbcd3679ed None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:00.630697 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f908c-1f3b-4605-afe2-45d22132b6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:00.632321 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f908c-1f3b-4605-afe2-45d22132b6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:00.632610 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f908c-1f3b-4605-afe2-45d22132b6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:00.633052 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f908c-1f3b-4605-afe2-45d22132b6d4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:00.643731 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f908c-1f3b-4605-afe2-45d22132b6d4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:00.645245 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f908c-1f3b-4605-afe2-45d22132b6d4 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:00.662846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-389f908c-1f3b-4605-afe2-45d22132b6d4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 19:31:00.662846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:00.662846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:00.662846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:00.662846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:00.662846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:00.662846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:00.662846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:00.662846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:00.662846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:00.662846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:00.662846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:00.662846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:31:00.662846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:31:00.662846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:31:00.662846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:31:00.663868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:00.663868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:00.663868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:31:00.663868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:00.663868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:31:00.663868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:00.663868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:00.663868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:00.663868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:00.663868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:00.663868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:00.663868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:00.663868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:00.663868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:00.663868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:00.663868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:00.663868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:00.663868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:00.670099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:00.670099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:00.670099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:31:00.670099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:31:00.670099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:31:00.670099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:31:00.670099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:31:00.670099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:31:00.670099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 19:31:00.670099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:00.670099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 327/655] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:31:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:00.670099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f908c-1f3b-4605-afe2-45d22132b6d4 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:00.712235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d1ac8-5b2f-44de-8d0b-7929024b9147 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:00.730250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:00.766486 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43c21a-2fb7-4584-96df-d10b0fe8ef24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:00.767303 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43c21a-2fb7-4584-96df-d10b0fe8ef24 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:00.767590 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43c21a-2fb7-4584-96df-d10b0fe8ef24 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:00.767870 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43c21a-2fb7-4584-96df-d10b0fe8ef24 None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:00.768690 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f43c21a-2fb7-4584-96df-d10b0fe8ef24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:00.769040 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f43c21a-2fb7-4584-96df-d10b0fe8ef24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:00.770733 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f43c21a-2fb7-4584-96df-d10b0fe8ef24 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:00.785392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 329/656] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:31:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:00.786701 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43c21a-2fb7-4584-96df-d10b0fe8ef24 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:00.809892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:00.816923 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f908c-1f3b-4605-afe2-45d22132b6d4 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:00.859472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b0f82-05a4-4c0a-9280-03d8b679b852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:00.860108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b0f82-05a4-4c0a-9280-03d8b679b852 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:00.860469 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b0f82-05a4-4c0a-9280-03d8b679b852 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:00.860754 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b0f82-05a4-4c0a-9280-03d8b679b852 None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:00.861755 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b0f82-05a4-4c0a-9280-03d8b679b852 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:00.862074 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b0f82-05a4-4c0a-9280-03d8b679b852 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:00.863644 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b0f82-05a4-4c0a-9280-03d8b679b852 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:00.888331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 328/657] 10.222.0.13 () {56 vars in 1139 bytes} [Fri May 27 19:31:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:00.888331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b0f82-05a4-4c0a-9280-03d8b679b852 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:00.921533 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f43c21a-2fb7-4584-96df-d10b0fe8ef24 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:00.966312 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496fa29-0032-4713-bf0d-d55666a05853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:00.967212 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496fa29-0032-4713-bf0d-d55666a05853 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:00.967506 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496fa29-0032-4713-bf0d-d55666a05853 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:00.967780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496fa29-0032-4713-bf0d-d55666a05853 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:00.979831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496fa29-0032-4713-bf0d-d55666a05853 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:00.981325 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496fa29-0032-4713-bf0d-d55666a05853 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:01.002190 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b0f82-05a4-4c0a-9280-03d8b679b852 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:01.007175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-c496fa29-0032-4713-bf0d-d55666a05853 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:01.007175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:01.007175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:01.007175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:01.007175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:01.007175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:01.007175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:01.007175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:01.007175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:01.007175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:01.007175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:01.007175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:01.007175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:01.007175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:01.007175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:01.008188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:01.008188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:01.008188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:01.008188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:01.008188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:01.008188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:01.008188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:01.008188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:01.008188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:01.008188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:01.008188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:01.008188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:01.008188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:01.008188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:01.008188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:01.008188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:01.008188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:01.009126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:01.009126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:01.009126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:01.009126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:01.009126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 330/658] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:01.009659 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496fa29-0032-4713-bf0d-d55666a05853 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:01.041575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:01.051613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc332da-427b-4424-ac94-66cba0caac52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:01.055303 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc332da-427b-4424-ac94-66cba0caac52 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:01.055303 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc332da-427b-4424-ac94-66cba0caac52 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:01.055303 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc332da-427b-4424-ac94-66cba0caac52 None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:01.055303 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc332da-427b-4424-ac94-66cba0caac52 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:01.055303 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc332da-427b-4424-ac94-66cba0caac52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:01.058652 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc332da-427b-4424-ac94-66cba0caac52 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:01.076144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 329/659] 10.222.0.13 () {56 vars in 1137 bytes} [Fri May 27 19:31:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:01.077032 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc332da-427b-4424-ac94-66cba0caac52 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:01.093414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:01.143883 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496fa29-0032-4713-bf0d-d55666a05853 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:01.180203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223741d-cedf-4f9d-a566-1e58b0b0718b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:01.180869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223741d-cedf-4f9d-a566-1e58b0b0718b None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:01.181178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223741d-cedf-4f9d-a566-1e58b0b0718b None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:01.181443 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223741d-cedf-4f9d-a566-1e58b0b0718b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:01.182278 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223741d-cedf-4f9d-a566-1e58b0b0718b None admin] RBAC: Adding query filter params (role.id=11e7404599ae49afb5edd22dabe1a5b2, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=7542e1fd160e4b2c9d25accf19aad19f) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:01.182585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223741d-cedf-4f9d-a566-1e58b0b0718b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:01.184019 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223741d-cedf-4f9d-a566-1e58b0b0718b None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:01.190749 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 331/660] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:31:01 2022] GET /identity/v3/role_assignments?user.id=7542e1fd160e4b2c9d25accf19aad19f&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=11e7404599ae49afb5edd22dabe1a5b2 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:01.192015 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116edf6c-56fd-4fb1-8f6d-94ea35cf9d2b None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:01.192304 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116edf6c-56fd-4fb1-8f6d-94ea35cf9d2b None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:01.193189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116edf6c-56fd-4fb1-8f6d-94ea35cf9d2b None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:01.200704 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 332/661] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:01.200704 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116edf6c-56fd-4fb1-8f6d-94ea35cf9d2b None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:01.201074 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc332da-427b-4424-ac94-66cba0caac52 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:01.258259 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f808bb-0468-476f-8014-e67cb224805f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:01.258948 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f808bb-0468-476f-8014-e67cb224805f None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:01.259232 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f808bb-0468-476f-8014-e67cb224805f None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:01.259511 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f808bb-0468-476f-8014-e67cb224805f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:01.319434 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:01.330274 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116edf6c-56fd-4fb1-8f6d-94ea35cf9d2b None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:01.356387 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f808bb-0468-476f-8014-e67cb224805f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:01.358127 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f808bb-0468-476f-8014-e67cb224805f None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:01.376039 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:01.382157 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cb956-3e54-4626-a9af-cab2a2786877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:01.383041 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cb956-3e54-4626-a9af-cab2a2786877 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:01.383427 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cb956-3e54-4626-a9af-cab2a2786877 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:01.383809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3cb956-3e54-4626-a9af-cab2a2786877 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:01.384822 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3cb956-3e54-4626-a9af-cab2a2786877 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:01.393179 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3cb956-3e54-4626-a9af-cab2a2786877 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:01.393179 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3cb956-3e54-4626-a9af-cab2a2786877 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:01.403987 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 333/662] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:01.403987 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cb956-3e54-4626-a9af-cab2a2786877 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:01.506586 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 330/663] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:31:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:31:01.508601 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b279888-4935-43ce-8be7-0632d18a357f None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:01.509180 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b279888-4935-43ce-8be7-0632d18a357f None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:01.509686 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b279888-4935-43ce-8be7-0632d18a357f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:01.530205 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-7b279888-4935-43ce-8be7-0632d18a357f None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:01.531424 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-7b279888-4935-43ce-8be7-0632d18a357f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:01.533242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3cb956-3e54-4626-a9af-cab2a2786877 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:01.574435 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd0882-7930-41c7-986c-22ba9b73263a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:01.575101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd0882-7930-41c7-986c-22ba9b73263a None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:01.575385 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd0882-7930-41c7-986c-22ba9b73263a None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:01.575659 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd0882-7930-41c7-986c-22ba9b73263a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:01.603439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:01.617220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 331/664] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:31:01.617220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b279888-4935-43ce-8be7-0632d18a357f None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:01.673984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd0882-7930-41c7-986c-22ba9b73263a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:01.675639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:01.676745 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd0882-7930-41c7-986c-22ba9b73263a None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:01.764087 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b279888-4935-43ce-8be7-0632d18a357f None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:01.809670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9cc45-e315-4c43-8c52-7d4a68f6b454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:01.811412 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9cc45-e315-4c43-8c52-7d4a68f6b454 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:01.811940 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9cc45-e315-4c43-8c52-7d4a68f6b454 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:01.812474 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9cc45-e315-4c43-8c52-7d4a68f6b454 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:01.824599 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9cc45-e315-4c43-8c52-7d4a68f6b454 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:01.826581 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf9cc45-e315-4c43-8c52-7d4a68f6b454 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:01.836993 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 334/665] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:31:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:31:01.836993 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd0882-7930-41c7-986c-22ba9b73263a None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:01.844988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-4cf9cc45-e315-4c43-8c52-7d4a68f6b454 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:01.844988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:01.844988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:01.844988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:01.844988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:01.844988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:01.844988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:01.844988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:01.844988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:01.844988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:01.844988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:01.844988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:01.844988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:31:01.844988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:31:01.844988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:31:01.846055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:31:01.846055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:01.846055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:01.846055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:01.846055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:01.846055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:01.846055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:01.846055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:01.846055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:01.846055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:01.846055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:01.846055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:01.846055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:01.846055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:01.846055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:01.846055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:01.846055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:01.847191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:31:01.847191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:31:01.847191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:31:01.847191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:31:01.847191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:31:01.847191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:31:01.847191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:01.847191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:01.847191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 332/666] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:01.847191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9cc45-e315-4c43-8c52-7d4a68f6b454 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:01.895447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:01.971369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd0882-7930-41c7-986c-22ba9b73263a None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:01.973130 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf9cc45-e315-4c43-8c52-7d4a68f6b454 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:01.975508 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:02.010168 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1cf46-83d4-4cfc-bd3c-5e54a309dd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:02.011118 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95f976-0f78-4591-b198-902cc5a28ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:02.011371 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1cf46-83d4-4cfc-bd3c-5e54a309dd4d None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:02.011853 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1cf46-83d4-4cfc-bd3c-5e54a309dd4d None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:02.012557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1cf46-83d4-4cfc-bd3c-5e54a309dd4d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:02.012986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95f976-0f78-4591-b198-902cc5a28ddc None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:02.013414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95f976-0f78-4591-b198-902cc5a28ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:02.013854 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95f976-0f78-4591-b198-902cc5a28ddc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:02.026672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b1cf46-83d4-4cfc-bd3c-5e54a309dd4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:31:02.033268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95f976-0f78-4591-b198-902cc5a28ddc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:02.035541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-b3b1cf46-83d4-4cfc-bd3c-5e54a309dd4d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 19:31:02.035541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:02.035541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:02.035541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:02.035541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:02.035541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:02.035541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:02.035541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:02.035541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:02.035541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:02.035541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:02.035541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:02.035541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:31:02.035541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:31:02.035541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:31:02.035541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:31:02.045650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:31:02.045650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:31:02.045650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:31:02.045650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:31:02.045650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:02.045650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:02.045650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:02.045650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:02.045650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:02.045650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:02.045650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:02.045650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:02.045650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:02.045650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:02.045650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:02.045650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:02.046596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:02.046596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:02.046596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:02.046596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:02.046596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:31:02.046596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:31:02.046596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:31:02.046596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:31:02.046596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:31:02.046596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:31:02.046596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 19:31:02.046596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:02.046596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 333/667] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:31:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:02.047452 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95f976-0f78-4591-b198-902cc5a28ddc None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:02.047574 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1cf46-83d4-4cfc-bd3c-5e54a309dd4d None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:02.060726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-ac95f976-0f78-4591-b198-902cc5a28ddc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:02.060726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:02.060726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:02.060726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:02.060726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:02.060726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:02.060726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:02.060726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:02.060726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:02.060726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:02.060726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:02.060726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:02.060726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:02.060726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:02.060726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:02.061986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:02.061986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:02.061986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:02.061986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:02.061986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:02.061986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:02.061986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:02.061986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:02.061986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:02.061986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:02.061986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:02.061986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:02.061986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:02.061986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:02.061986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:02.061986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:02.061986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:02.063004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:02.063004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:02.063004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:02.063004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:02.063004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 335/668] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:02.063296 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95f976-0f78-4591-b198-902cc5a28ddc None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:02.164787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1cf46-83d4-4cfc-bd3c-5e54a309dd4d None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:02.183556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95f976-0f78-4591-b198-902cc5a28ddc None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:02.185277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:02.202626 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f0ae8-ebbf-42bf-8e9e-93ab5f60df88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:02.203429 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f0ae8-ebbf-42bf-8e9e-93ab5f60df88 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:02.203710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f0ae8-ebbf-42bf-8e9e-93ab5f60df88 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:02.203982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f0ae8-ebbf-42bf-8e9e-93ab5f60df88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:02.216364 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f0ae8-ebbf-42bf-8e9e-93ab5f60df88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:02.217870 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f0ae8-ebbf-42bf-8e9e-93ab5f60df88 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:02.220783 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d6ceb-2666-4c24-8a1e-04b4654745e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:02.222096 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d6ceb-2666-4c24-8a1e-04b4654745e5 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:02.222737 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d6ceb-2666-4c24-8a1e-04b4654745e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:02.223689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d6ceb-2666-4c24-8a1e-04b4654745e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:02.225376 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d6ceb-2666-4c24-8a1e-04b4654745e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:02.225695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d6ceb-2666-4c24-8a1e-04b4654745e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:02.227265 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d6ceb-2666-4c24-8a1e-04b4654745e5 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:02.235749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-171f0ae8-ebbf-42bf-8e9e-93ab5f60df88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:02.235749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:02.235749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:02.235749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:02.235749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:02.235749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:02.235749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:02.235749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:02.235749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:02.235749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:02.235749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:02.235749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:02.235749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:02.235749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:02.235749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:02.237037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:02.237037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:02.237037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:02.237037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:02.237037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:02.237037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:02.237037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:02.237037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:02.237037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:02.237037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:02.237037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:02.237037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:02.237037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:02.237037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:02.237037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:02.237037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:02.237037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:02.239718 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:02.239718 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:02.239718 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:02.239718 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:02.239718 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 334/669] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:02.239718 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f0ae8-ebbf-42bf-8e9e-93ab5f60df88 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:02.247579 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 336/670] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:31:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:02.248847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d6ceb-2666-4c24-8a1e-04b4654745e5 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:02.257649 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:02.350054 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f0ae8-ebbf-42bf-8e9e-93ab5f60df88 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:02.367255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d6ceb-2666-4c24-8a1e-04b4654745e5 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:02.391506 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db775e4c-f530-42a6-9094-059d71ac640f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:02.392209 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db775e4c-f530-42a6-9094-059d71ac640f None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:02.392494 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db775e4c-f530-42a6-9094-059d71ac640f None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:02.392768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db775e4c-f530-42a6-9094-059d71ac640f None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:02.393548 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db775e4c-f530-42a6-9094-059d71ac640f None admin] RBAC: Adding query filter params (name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:02.393853 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db775e4c-f530-42a6-9094-059d71ac640f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:02.395234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db775e4c-f530-42a6-9094-059d71ac640f None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:02.403762 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a9bdc-76db-4fd2-b419-cc3ff1c716f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:02.409170 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 335/671] 10.222.0.13 () {56 vars in 1103 bytes} [Fri May 27 19:31:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:02.409257 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a9bdc-76db-4fd2-b419-cc3ff1c716f3 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:02.409257 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a9bdc-76db-4fd2-b419-cc3ff1c716f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:02.409423 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db775e4c-f530-42a6-9094-059d71ac640f None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:02.409505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a9bdc-76db-4fd2-b419-cc3ff1c716f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:02.409505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a9bdc-76db-4fd2-b419-cc3ff1c716f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:02.409505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a9bdc-76db-4fd2-b419-cc3ff1c716f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:02.409505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a9bdc-76db-4fd2-b419-cc3ff1c716f3 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:02.421705 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 337/672] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:02.427328 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a9bdc-76db-4fd2-b419-cc3ff1c716f3 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:02.454090 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:02.495994 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:02.524892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db775e4c-f530-42a6-9094-059d71ac640f None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:02.547376 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a9bdc-76db-4fd2-b419-cc3ff1c716f3 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:02.563612 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efce702-8bfc-4990-93fb-93ad3bad3ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:02.564291 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efce702-8bfc-4990-93fb-93ad3bad3ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:02.564576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efce702-8bfc-4990-93fb-93ad3bad3ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:02.564856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efce702-8bfc-4990-93fb-93ad3bad3ac2 None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:02.565619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efce702-8bfc-4990-93fb-93ad3bad3ac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:02.565928 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efce702-8bfc-4990-93fb-93ad3bad3ac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:02.567848 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efce702-8bfc-4990-93fb-93ad3bad3ac2 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:02.582564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 336/673] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:02.583496 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efce702-8bfc-4990-93fb-93ad3bad3ac2 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:02.592036 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02102ab2-2469-4735-a556-f6769d993184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:02.593074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02102ab2-2469-4735-a556-f6769d993184 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:02.593559 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02102ab2-2469-4735-a556-f6769d993184 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:02.594029 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02102ab2-2469-4735-a556-f6769d993184 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:02.595570 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02102ab2-2469-4735-a556-f6769d993184 None admin] RBAC: Adding query filter params (role.id=11e7404599ae49afb5edd22dabe1a5b2, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=1347c0bd4ada4139ad194d423958be6e) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:02.596429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02102ab2-2469-4735-a556-f6769d993184 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:02.599523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02102ab2-2469-4735-a556-f6769d993184 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:02.617010 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 338/674] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:31:02 2022] GET /identity/v3/role_assignments?user.id=1347c0bd4ada4139ad194d423958be6e&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=11e7404599ae49afb5edd22dabe1a5b2 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:02.617010 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02102ab2-2469-4735-a556-f6769d993184 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:02.729715 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:02.730570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efce702-8bfc-4990-93fb-93ad3bad3ac2 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:02.753582 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02102ab2-2469-4735-a556-f6769d993184 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:02.773703 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480465a-a6ed-435a-89f1-c010a55af387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:02.774713 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480465a-a6ed-435a-89f1-c010a55af387 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:02.775306 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480465a-a6ed-435a-89f1-c010a55af387 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:02.775596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480465a-a6ed-435a-89f1-c010a55af387 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:02.779118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:02.796002 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480465a-a6ed-435a-89f1-c010a55af387 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:02.797817 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480465a-a6ed-435a-89f1-c010a55af387 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:02.808939 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903492b-6e37-4ebf-a7ce-67c8b1f87ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:02.810092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903492b-6e37-4ebf-a7ce-67c8b1f87ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:02.810475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903492b-6e37-4ebf-a7ce-67c8b1f87ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:02.810854 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c903492b-6e37-4ebf-a7ce-67c8b1f87ba5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:02.819854 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903492b-6e37-4ebf-a7ce-67c8b1f87ba5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:02.821741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c903492b-6e37-4ebf-a7ce-67c8b1f87ba5 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:02.823319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-c480465a-a6ed-435a-89f1-c010a55af387 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:02.823319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:02.823319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:02.823319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:02.823319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:02.823319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:02.823319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:02.823319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:02.823319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:02.823319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:02.823319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:02.823319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:02.823319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:02.823319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:02.823319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:02.824320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:02.824320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:02.824320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:02.824320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:02.824320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:02.824320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:02.824320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:02.824320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:02.824320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:02.824320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:02.824320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:02.824320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:02.824320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:02.824320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:02.824320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:02.824320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:02.824320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:02.825265 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:02.825265 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:02.825265 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:02.825265 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:02.832701 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 337/675] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:02.834269 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480465a-a6ed-435a-89f1-c010a55af387 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:02.842614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-c903492b-6e37-4ebf-a7ce-67c8b1f87ba5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:02.842614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:02.842614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:02.842614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:02.842614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:02.842614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:02.842614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:02.842614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:02.842614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:02.842614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:02.842614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:02.842614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:02.842614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:31:02.842614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:31:02.842614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:31:02.843734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:31:02.843734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:02.843734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:02.843734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:02.843734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:02.843734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:02.843734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:02.843734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:02.843734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:02.843734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:02.843734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:02.843734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:02.843734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:02.843734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:02.843734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:02.843734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:02.843734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:02.844705 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:31:02.844705 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:31:02.844705 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:31:02.844705 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:31:02.844705 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:31:02.844705 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:31:02.844705 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:02.844705 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:02.844705 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 339/676] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:02.845701 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903492b-6e37-4ebf-a7ce-67c8b1f87ba5 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:02.980939 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c903492b-6e37-4ebf-a7ce-67c8b1f87ba5 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:02.984305 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480465a-a6ed-435a-89f1-c010a55af387 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:03.028044 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987b5ae-e24b-46e9-aa74-d68ae42e69e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:03.029036 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987b5ae-e24b-46e9-aa74-d68ae42e69e9 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:03.029415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987b5ae-e24b-46e9-aa74-d68ae42e69e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:03.029741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9987b5ae-e24b-46e9-aa74-d68ae42e69e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:03.030764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987b5ae-e24b-46e9-aa74-d68ae42e69e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:03.031149 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987b5ae-e24b-46e9-aa74-d68ae42e69e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:03.032818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9987b5ae-e24b-46e9-aa74-d68ae42e69e9 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:03.033473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda68078-898b-4d68-aee1-6ebdbb7c8ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:03.034717 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda68078-898b-4d68-aee1-6ebdbb7c8ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:03.035155 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda68078-898b-4d68-aee1-6ebdbb7c8ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:03.036344 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda68078-898b-4d68-aee1-6ebdbb7c8ac0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:03.045727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda68078-898b-4d68-aee1-6ebdbb7c8ac0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:03.047436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda68078-898b-4d68-aee1-6ebdbb7c8ac0 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:03.047650 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 340/677] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:03.052874 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987b5ae-e24b-46e9-aa74-d68ae42e69e9 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:03.060814 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:03.073562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-bda68078-898b-4d68-aee1-6ebdbb7c8ac0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:03.073562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:03.073562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:03.073562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:03.073562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:03.073562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:03.073562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:03.073562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:03.073562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:03.073562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:03.073562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:03.073562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:03.073562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:31:03.073562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:31:03.073562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:31:03.081357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:31:03.081357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:03.081357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:03.081357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:03.081357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:03.081357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:03.081357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:03.081357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:03.081357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:03.081357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:03.081357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:03.081357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:03.081357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:03.081357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:03.081357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:03.081357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:03.081357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:03.082512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:31:03.082512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:31:03.082512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:31:03.082512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:31:03.082512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:31:03.082512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:31:03.082512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:03.082512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:03.082512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 338/678] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:03.082512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda68078-898b-4d68-aee1-6ebdbb7c8ac0 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:03.149040 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:03.185653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9987b5ae-e24b-46e9-aa74-d68ae42e69e9 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:03.222465 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda68078-898b-4d68-aee1-6ebdbb7c8ac0 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:03.236247 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e638498-660b-4732-aa54-2c4edd9f25f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:03.236473 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e638498-660b-4732-aa54-2c4edd9f25f9 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:03.236728 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e638498-660b-4732-aa54-2c4edd9f25f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:03.237050 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e638498-660b-4732-aa54-2c4edd9f25f9 None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:03.237848 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e638498-660b-4732-aa54-2c4edd9f25f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:03.238163 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e638498-660b-4732-aa54-2c4edd9f25f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:03.240840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e638498-660b-4732-aa54-2c4edd9f25f9 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:03.258891 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 341/679] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:31:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:03.259932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e638498-660b-4732-aa54-2c4edd9f25f9 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:03.270482 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df603b80-f424-4600-ba38-125e5d3718a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:03.272175 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df603b80-f424-4600-ba38-125e5d3718a4 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:03.272674 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df603b80-f424-4600-ba38-125e5d3718a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:03.273601 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df603b80-f424-4600-ba38-125e5d3718a4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:03.283995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df603b80-f424-4600-ba38-125e5d3718a4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:03.286814 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df603b80-f424-4600-ba38-125e5d3718a4 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:03.305068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-df603b80-f424-4600-ba38-125e5d3718a4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 19:31:03.305068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:03.305068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:03.305068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:03.305068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:03.305068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:03.305068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:03.305068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:03.305068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:03.305068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:03.305068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:03.305068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:03.305068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:31:03.305068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:31:03.305068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:31:03.305068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:31:03.306090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:03.306090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:03.306090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:31:03.306090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:03.306090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:31:03.306090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:03.306090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:03.306090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:03.306090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:03.306090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:03.306090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:03.306090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:03.306090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:03.306090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:03.306090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:03.306090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:03.306090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:03.306090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:03.307267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:03.307267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:03.307267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:31:03.307267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:31:03.307267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:31:03.307267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:31:03.307267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:31:03.307267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:31:03.307267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 19:31:03.307267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:03.307267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 339/680] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:31:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:03.307900 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df603b80-f424-4600-ba38-125e5d3718a4 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:03.345453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:03.382560 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e638498-660b-4732-aa54-2c4edd9f25f9 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:03.420899 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a069b6-df59-4370-9b72-be7e323bb7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:03.422396 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a069b6-df59-4370-9b72-be7e323bb7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:03.422879 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a069b6-df59-4370-9b72-be7e323bb7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:03.423340 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a069b6-df59-4370-9b72-be7e323bb7e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:03.424339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a069b6-df59-4370-9b72-be7e323bb7e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:03.424861 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a069b6-df59-4370-9b72-be7e323bb7e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:03.426716 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a069b6-df59-4370-9b72-be7e323bb7e3 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:03.428559 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df603b80-f424-4600-ba38-125e5d3718a4 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:03.439960 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:03.452382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 342/681] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:31:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:03.455411 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a069b6-df59-4370-9b72-be7e323bb7e3 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:03.472083 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271454b3-f361-454f-8d7c-29834aaf1f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:03.472751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271454b3-f361-454f-8d7c-29834aaf1f29 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:03.473059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271454b3-f361-454f-8d7c-29834aaf1f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:03.473873 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271454b3-f361-454f-8d7c-29834aaf1f29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:03.474718 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271454b3-f361-454f-8d7c-29834aaf1f29 None admin] RBAC: Adding query filter params (role.id=11e7404599ae49afb5edd22dabe1a5b2, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=dd34f15400c4476bbb2161dbe1b8b2b3) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:03.475026 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271454b3-f361-454f-8d7c-29834aaf1f29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:03.476977 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271454b3-f361-454f-8d7c-29834aaf1f29 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:03.484362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 340/682] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:31:03 2022] GET /identity/v3/role_assignments?user.id=dd34f15400c4476bbb2161dbe1b8b2b3&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=11e7404599ae49afb5edd22dabe1a5b2 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:03.485317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271454b3-f361-454f-8d7c-29834aaf1f29 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:03.573000 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a069b6-df59-4370-9b72-be7e323bb7e3 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:03.613323 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:03.633796 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a022d-0a2a-49e0-ab76-cfbf623693ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:03.649281 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a022d-0a2a-49e0-ab76-cfbf623693ac None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:03.649281 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a022d-0a2a-49e0-ab76-cfbf623693ac None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:03.649281 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a022d-0a2a-49e0-ab76-cfbf623693ac None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:03.649281 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a022d-0a2a-49e0-ab76-cfbf623693ac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:03.649281 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a022d-0a2a-49e0-ab76-cfbf623693ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:03.649281 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a022d-0a2a-49e0-ab76-cfbf623693ac None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:03.649768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271454b3-f361-454f-8d7c-29834aaf1f29 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:03.670051 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 343/683] 10.222.0.13 () {56 vars in 1135 bytes} [Fri May 27 19:31:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:03.671979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc22023-a858-4898-998b-619de3cbff2a None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:03.672494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc22023-a858-4898-998b-619de3cbff2a None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:03.673017 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc22023-a858-4898-998b-619de3cbff2a None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:03.675185 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 344/684] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:03.676966 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc22023-a858-4898-998b-619de3cbff2a None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:03.688700 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50123f51-bbff-46a3-9173-6765952eabd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:03.689111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50123f51-bbff-46a3-9173-6765952eabd5 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:03.689380 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50123f51-bbff-46a3-9173-6765952eabd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:03.689660 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50123f51-bbff-46a3-9173-6765952eabd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:03.690513 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50123f51-bbff-46a3-9173-6765952eabd5 None admin] RBAC: Adding query filter params (role.id=11e7404599ae49afb5edd22dabe1a5b2, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=7b7e90ca75d643cbb0e6d6791fc30b45) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:03.690825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50123f51-bbff-46a3-9173-6765952eabd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:03.693452 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50123f51-bbff-46a3-9173-6765952eabd5 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:03.700333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 341/685] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:31:03 2022] GET /identity/v3/role_assignments?user.id=7b7e90ca75d643cbb0e6d6791fc30b45&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=11e7404599ae49afb5edd22dabe1a5b2 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:03.709012 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199ad32-5fa8-482e-9f4f-328339a2efe4 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:03.709012 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199ad32-5fa8-482e-9f4f-328339a2efe4 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:03.709012 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2199ad32-5fa8-482e-9f4f-328339a2efe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:03.724981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-2199ad32-5fa8-482e-9f4f-328339a2efe4 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:03.724981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-2199ad32-5fa8-482e-9f4f-328339a2efe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:03.760393 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:03.814872 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 342/686] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:31:03.840537 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199ad32-5fa8-482e-9f4f-328339a2efe4 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:03.859847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc22023-a858-4898-998b-619de3cbff2a None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:03.910415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f06dc8-2a55-4b3e-a80f-8d73f0a737dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:03.913726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f06dc8-2a55-4b3e-a80f-8d73f0a737dd None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:03.913726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f06dc8-2a55-4b3e-a80f-8d73f0a737dd None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:03.914035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f06dc8-2a55-4b3e-a80f-8d73f0a737dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:03.998870 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:04.017052 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2199ad32-5fa8-482e-9f4f-328339a2efe4 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:04.029888 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f06dc8-2a55-4b3e-a80f-8d73f0a737dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:04.032182 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f06dc8-2a55-4b3e-a80f-8d73f0a737dd None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:04.056575 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403ce9a-b564-4d79-974d-e73275828afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:04.057444 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403ce9a-b564-4d79-974d-e73275828afc None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:04.057725 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403ce9a-b564-4d79-974d-e73275828afc None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:04.058000 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403ce9a-b564-4d79-974d-e73275828afc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:04.070982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b403ce9a-b564-4d79-974d-e73275828afc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:31:04.079122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-b403ce9a-b564-4d79-974d-e73275828afc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:31:04.079122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:04.079122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:04.079122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:04.079122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:04.079122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:04.079122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:04.079122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:04.079122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:04.079122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:04.079122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:04.079122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:04.079122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:31:04.079122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:31:04.079122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:31:04.079122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:31:04.080409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:31:04.080409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:31:04.080409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:31:04.080409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:31:04.080409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:04.080409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:04.080409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:04.080409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:04.080409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:04.080409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:04.080409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:04.080409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:04.080409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:04.080409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:04.080409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:04.080409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:04.081557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:04.081557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:04.081557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:04.081557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:04.081557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:31:04.081557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:31:04.081557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:31:04.081557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:31:04.081557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:31:04.081557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:31:04.081557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:31:04.081557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:04.081557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 343/687] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:31:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:04.086389 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403ce9a-b564-4d79-974d-e73275828afc None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:04.118066 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:04.156907 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 345/688] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:31:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:31:04.163437 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f06dc8-2a55-4b3e-a80f-8d73f0a737dd None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:04.264813 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403ce9a-b564-4d79-974d-e73275828afc None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:04.274345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:04.304030 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f06dc8-2a55-4b3e-a80f-8d73f0a737dd None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:04.323500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119eee8-2765-4e49-8a35-0c18adbf4c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:04.324412 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119eee8-2765-4e49-8a35-0c18adbf4c7f None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:04.324412 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119eee8-2765-4e49-8a35-0c18adbf4c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:04.324412 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119eee8-2765-4e49-8a35-0c18adbf4c7f None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:04.325337 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119eee8-2765-4e49-8a35-0c18adbf4c7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:04.325894 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119eee8-2765-4e49-8a35-0c18adbf4c7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:04.327517 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119eee8-2765-4e49-8a35-0c18adbf4c7f None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:04.342230 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 344/689] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:31:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:31:04.347617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119eee8-2765-4e49-8a35-0c18adbf4c7f None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:04.378877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592d99b-4a51-41b4-80f5-c8eb39958e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:04.380696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592d99b-4a51-41b4-80f5-c8eb39958e9f None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:04.383960 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592d99b-4a51-41b4-80f5-c8eb39958e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:04.384460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592d99b-4a51-41b4-80f5-c8eb39958e9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:04.413253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592d99b-4a51-41b4-80f5-c8eb39958e9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:04.414984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592d99b-4a51-41b4-80f5-c8eb39958e9f None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:04.441927 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-d592d99b-4a51-41b4-80f5-c8eb39958e9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:04.441927 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:04.441927 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:04.441927 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:04.441927 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:04.441927 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:04.441927 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:04.441927 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:04.441927 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:04.441927 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:04.441927 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:04.441927 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:04.441927 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:04.441927 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:04.441927 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:04.443526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:04.443526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:04.443526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:04.443526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:04.443526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:04.443526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:04.443526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:04.443526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:04.443526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:04.443526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:04.443526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:04.443526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:04.443526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:04.443526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:04.443526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:04.443526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:04.443526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:04.444462 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:04.444462 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:04.444462 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:04.444462 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:04.447095 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 346/690] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:04.456703 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592d99b-4a51-41b4-80f5-c8eb39958e9f None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:04.509964 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119eee8-2765-4e49-8a35-0c18adbf4c7f None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:04.520781 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:04.552796 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd160f3-aa0b-453d-a6db-a5becf550ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:04.553801 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd160f3-aa0b-453d-a6db-a5becf550ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:04.554065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd160f3-aa0b-453d-a6db-a5becf550ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:04.554339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd160f3-aa0b-453d-a6db-a5becf550ce6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:04.566686 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd160f3-aa0b-453d-a6db-a5becf550ce6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:04.570591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd160f3-aa0b-453d-a6db-a5becf550ce6 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:04.592523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-4cd160f3-aa0b-453d-a6db-a5becf550ce6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:04.592523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:04.592523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:04.592523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:04.592523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:04.592523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:04.592523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:04.592523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:04.592523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:04.592523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:04.592523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:04.592523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:04.592523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:04.592523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:04.592523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:04.593532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:04.593532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:04.593532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:04.593532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:04.593532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:04.593532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:04.593532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:04.593532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:04.593532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:04.593532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:04.593532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:04.593532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:04.593532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:04.593532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:04.593532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:04.593532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:04.593532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:04.594527 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:04.594527 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:04.594527 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:04.594527 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:04.594527 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 345/691] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:31:04.604115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd160f3-aa0b-453d-a6db-a5becf550ce6 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:04.760122 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592d99b-4a51-41b4-80f5-c8eb39958e9f None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:04.782018 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd160f3-aa0b-453d-a6db-a5becf550ce6 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:04.815135 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209c639-1de7-4c79-a135-78fa54f43c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:04.816229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209c639-1de7-4c79-a135-78fa54f43c36 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:04.816749 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209c639-1de7-4c79-a135-78fa54f43c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:04.819910 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209c639-1de7-4c79-a135-78fa54f43c36 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:04.820921 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209c639-1de7-4c79-a135-78fa54f43c36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:04.821537 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209c639-1de7-4c79-a135-78fa54f43c36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:04.823168 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209c639-1de7-4c79-a135-78fa54f43c36 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:04.827933 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:04.842375 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 347/692] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:04.849968 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209c639-1de7-4c79-a135-78fa54f43c36 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:04.878260 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43dc63e-3679-4c13-9515-c4080fce7a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:04.879351 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43dc63e-3679-4c13-9515-c4080fce7a95 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:04.879860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43dc63e-3679-4c13-9515-c4080fce7a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:04.880470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43dc63e-3679-4c13-9515-c4080fce7a95 None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:04.881468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43dc63e-3679-4c13-9515-c4080fce7a95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:04.881999 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43dc63e-3679-4c13-9515-c4080fce7a95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:04.883606 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43dc63e-3679-4c13-9515-c4080fce7a95 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:04.904296 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:04.913842 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 346/693] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:04.920110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43dc63e-3679-4c13-9515-c4080fce7a95 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:05.086796 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209c639-1de7-4c79-a135-78fa54f43c36 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:05.157160 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43dc63e-3679-4c13-9515-c4080fce7a95 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:05.163135 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1a1f2-6f13-4b44-a7b1-d4b45afdb707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:05.169146 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1a1f2-6f13-4b44-a7b1-d4b45afdb707 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:05.169730 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1a1f2-6f13-4b44-a7b1-d4b45afdb707 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:05.170219 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba1a1f2-6f13-4b44-a7b1-d4b45afdb707 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:05.183047 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1a1f2-6f13-4b44-a7b1-d4b45afdb707 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:05.188036 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba1a1f2-6f13-4b44-a7b1-d4b45afdb707 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:05.217215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-8ba1a1f2-6f13-4b44-a7b1-d4b45afdb707 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:05.217215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:05.217215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:05.217215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:05.217215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:05.217215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:05.217215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:05.217215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:05.217215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:05.217215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:05.217215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:05.217215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:05.217215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:31:05.217215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:31:05.217215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:31:05.218217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:31:05.218217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:05.218217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:05.218217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:05.218217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:05.218217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:05.218217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:05.218217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:05.218217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:05.218217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:05.218217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:05.218217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:05.218217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:05.218217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:05.218217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:05.218217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:05.218217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:05.219133 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:31:05.219133 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:31:05.219133 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:31:05.219133 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:31:05.219133 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:31:05.219133 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:31:05.219133 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:05.219133 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:05.227921 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 348/694] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:05.236680 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c90bc45-9e4b-45e6-8938-ba8039601639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:05.237531 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90bc45-9e4b-45e6-8938-ba8039601639 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:05.237816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90bc45-9e4b-45e6-8938-ba8039601639 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:05.238094 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90bc45-9e4b-45e6-8938-ba8039601639 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:05.239492 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1a1f2-6f13-4b44-a7b1-d4b45afdb707 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:05.270394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c90bc45-9e4b-45e6-8938-ba8039601639 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:05.272253 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c90bc45-9e4b-45e6-8938-ba8039601639 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:05.317640 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:05.322131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-1c90bc45-9e4b-45e6-8938-ba8039601639 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 19:31:05.322131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:05.322131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:05.322131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:05.322131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:05.322131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:05.322131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:05.322131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:05.322131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:05.322131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:05.322131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:05.322131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:05.322131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:31:05.322131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:31:05.322131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:31:05.322131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:31:05.323121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:05.323121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:05.323121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:31:05.323121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:05.323121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:31:05.323121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:05.323121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:05.323121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:05.323121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:05.323121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:05.323121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:05.323121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:05.323121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:05.323121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:05.323121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:05.323121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:05.323121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:05.323121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:05.324101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:05.324101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:05.324101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:31:05.324101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:31:05.324101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:31:05.324101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:31:05.324101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:31:05.324101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:31:05.324101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 19:31:05.324101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:05.324101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 347/695] 10.222.0.13 () {56 vars in 1093 bytes} [Fri May 27 19:31:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:31:05.334310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c90bc45-9e4b-45e6-8938-ba8039601639 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:05.377539 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba1a1f2-6f13-4b44-a7b1-d4b45afdb707 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:05.417090 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ab9bc-45c9-49d7-b0b7-f88d9b58ca7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:05.418256 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ab9bc-45c9-49d7-b0b7-f88d9b58ca7f None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:05.418781 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ab9bc-45c9-49d7-b0b7-f88d9b58ca7f None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:05.419390 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9ab9bc-45c9-49d7-b0b7-f88d9b58ca7f None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:05.420392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ab9bc-45c9-49d7-b0b7-f88d9b58ca7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:05.420908 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ab9bc-45c9-49d7-b0b7-f88d9b58ca7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:05.422612 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9ab9bc-45c9-49d7-b0b7-f88d9b58ca7f None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:05.438097 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 349/696] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:31:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:05.444588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ab9bc-45c9-49d7-b0b7-f88d9b58ca7f None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:05.504556 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:05.616170 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c90bc45-9e4b-45e6-8938-ba8039601639 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:05.671570 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9ab9bc-45c9-49d7-b0b7-f88d9b58ca7f None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:05.693140 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5dd75-d000-4f44-ba21-438296a1e5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:05.693888 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5dd75-d000-4f44-ba21-438296a1e5dd None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:05.694175 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5dd75-d000-4f44-ba21-438296a1e5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:05.696990 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5dd75-d000-4f44-ba21-438296a1e5dd None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:05.696990 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a5dd75-d000-4f44-ba21-438296a1e5dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:05.696990 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a5dd75-d000-4f44-ba21-438296a1e5dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:05.697257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a5dd75-d000-4f44-ba21-438296a1e5dd None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:05.712542 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:05.745204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 348/697] 10.222.0.13 () {56 vars in 1133 bytes} [Fri May 27 19:31:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:05.748612 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76a0e28-57de-4122-b747-2a40d784dbcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:05.749321 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a0e28-57de-4122-b747-2a40d784dbcc None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:05.749605 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a0e28-57de-4122-b747-2a40d784dbcc None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:05.749881 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76a0e28-57de-4122-b747-2a40d784dbcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:05.750729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a0e28-57de-4122-b747-2a40d784dbcc None admin] RBAC: Adding query filter params (role.id=11e7404599ae49afb5edd22dabe1a5b2, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=37bc6d0abb454de19e1ab2f46bf4a593) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:05.751109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a0e28-57de-4122-b747-2a40d784dbcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:05.752682 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5dd75-d000-4f44-ba21-438296a1e5dd None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:05.758469 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76a0e28-57de-4122-b747-2a40d784dbcc None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:05.775103 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 350/698] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:31:05 2022] GET /identity/v3/role_assignments?user.id=37bc6d0abb454de19e1ab2f46bf4a593&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=11e7404599ae49afb5edd22dabe1a5b2 => generated 608 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:31:05.822770 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2995c-4248-434d-acca-380ed26a11f6 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:05.823122 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2995c-4248-434d-acca-380ed26a11f6 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:05.825077 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2995c-4248-434d-acca-380ed26a11f6 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:05.825077 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 351/699] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:05.834931 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c71edc-a562-442e-b726-a45af00c6ef6 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:05.835601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c71edc-a562-442e-b726-a45af00c6ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:05.836078 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c71edc-a562-442e-b726-a45af00c6ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:05.877766 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-a4c71edc-a562-442e-b726-a45af00c6ef6 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:05.879790 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-a4c71edc-a562-442e-b726-a45af00c6ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:05.950321 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5dd75-d000-4f44-ba21-438296a1e5dd None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:05.978248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:06.000976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f96ec-07e7-48ff-a644-838cb06f7798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:06.001654 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f96ec-07e7-48ff-a644-838cb06f7798 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:06.001940 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f96ec-07e7-48ff-a644-838cb06f7798 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:06.002218 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f96ec-07e7-48ff-a644-838cb06f7798 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:06.103399 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 352/700] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:31:06.130163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f96ec-07e7-48ff-a644-838cb06f7798 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:06.131893 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f96ec-07e7-48ff-a644-838cb06f7798 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:06.138159 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c71edc-a562-442e-b726-a45af00c6ef6 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:06.275819 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 349/701] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:31:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 19:31:06.282670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f96ec-07e7-48ff-a644-838cb06f7798 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:06.300781 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c71edc-a562-442e-b726-a45af00c6ef6 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:06.317636 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:06.320104 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:06.354340 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7707b42-8c10-46f4-8a58-dbea7883a39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:06.356378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7707b42-8c10-46f4-8a58-dbea7883a39a None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:06.356914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7707b42-8c10-46f4-8a58-dbea7883a39a None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:06.357434 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7707b42-8c10-46f4-8a58-dbea7883a39a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:06.371893 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7707b42-8c10-46f4-8a58-dbea7883a39a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:06.374017 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7707b42-8c10-46f4-8a58-dbea7883a39a None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:06.398969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-f7707b42-8c10-46f4-8a58-dbea7883a39a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:06.398969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:06.398969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:06.398969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:06.398969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:06.398969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:06.398969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:06.398969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:06.398969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:06.398969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:06.398969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:06.398969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:06.398969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:06.398969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:06.398969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:06.399949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:06.399949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:06.399949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:06.399949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:06.399949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:06.399949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:06.399949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:06.399949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:06.399949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:06.399949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:06.399949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:06.399949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:06.399949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:06.399949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:06.399949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:06.399949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:06.399949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:06.400986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:06.400986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:06.400986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:06.400986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:06.403117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 353/702] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:06.413266 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7707b42-8c10-46f4-8a58-dbea7883a39a None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:06.449152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f96ec-07e7-48ff-a644-838cb06f7798 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:06.501523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b67eb6-8e2d-4fe7-b6f2-7e498f8a6259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:06.502366 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b67eb6-8e2d-4fe7-b6f2-7e498f8a6259 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:06.502654 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b67eb6-8e2d-4fe7-b6f2-7e498f8a6259 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:06.505419 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b67eb6-8e2d-4fe7-b6f2-7e498f8a6259 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:06.527837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b67eb6-8e2d-4fe7-b6f2-7e498f8a6259 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:06.529493 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b67eb6-8e2d-4fe7-b6f2-7e498f8a6259 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:06.557484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-00b67eb6-8e2d-4fe7-b6f2-7e498f8a6259 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:06.557484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:06.557484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:06.557484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:06.557484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:06.557484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:06.557484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:06.557484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:06.557484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:06.557484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:06.557484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:06.557484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:06.557484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:06.557484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:06.557484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:06.558473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:06.558473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:06.558473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:06.558473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:06.558473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:06.558473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:06.558473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:06.558473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:06.558473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:06.558473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:06.558473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:06.558473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:06.558473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:06.558473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:06.558473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:06.558473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:06.558473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:06.559392 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:06.559392 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:06.559392 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:06.559392 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:06.561042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 350/703] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:31:06.568979 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9726e6e-443c-458a-8181-7f56c6461347 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:06.568979 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9726e6e-443c-458a-8181-7f56c6461347 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:06.568979 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9726e6e-443c-458a-8181-7f56c6461347 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:06.568979 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 351/704] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:06.568979 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9726e6e-443c-458a-8181-7f56c6461347 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:06.572718 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7707b42-8c10-46f4-8a58-dbea7883a39a None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:06.652238 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f64a485-80da-4989-aa47-002cacc2bae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:06.652939 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64a485-80da-4989-aa47-002cacc2bae9 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:06.653357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64a485-80da-4989-aa47-002cacc2bae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:06.653637 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64a485-80da-4989-aa47-002cacc2bae9 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:06.654453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f64a485-80da-4989-aa47-002cacc2bae9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:06.654766 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f64a485-80da-4989-aa47-002cacc2bae9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:06.656453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f64a485-80da-4989-aa47-002cacc2bae9 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:06.676437 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 354/705] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:06.679682 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:06.680837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9ef33-dfa7-4b37-9209-da6618a42ad6 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:06.684425 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9ef33-dfa7-4b37-9209-da6618a42ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:06.684425 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d9ef33-dfa7-4b37-9209-da6618a42ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:06.713419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-93d9ef33-dfa7-4b37-9209-da6618a42ad6 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:06.715183 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-93d9ef33-dfa7-4b37-9209-da6618a42ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:06.718464 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9726e6e-443c-458a-8181-7f56c6461347 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:06.748751 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:06.778592 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04bedb-c9d6-4d5f-b6f8-13298661f657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:06.779634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04bedb-c9d6-4d5f-b6f8-13298661f657 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:06.780251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04bedb-c9d6-4d5f-b6f8-13298661f657 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:06.780935 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04bedb-c9d6-4d5f-b6f8-13298661f657 None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:06.784843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04bedb-c9d6-4d5f-b6f8-13298661f657 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:06.785860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04bedb-c9d6-4d5f-b6f8-13298661f657 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:06.789348 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e04bedb-c9d6-4d5f-b6f8-13298661f657 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:06.811647 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 352/706] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:06.813309 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04bedb-c9d6-4d5f-b6f8-13298661f657 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:06.881421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 355/707] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:31:06.882900 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9ef33-dfa7-4b37-9209-da6618a42ad6 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:06.963517 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04bedb-c9d6-4d5f-b6f8-13298661f657 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:06.998999 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:07.007213 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6677f-ac26-4e71-9834-3b445e4611ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:07.007909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6677f-ac26-4e71-9834-3b445e4611ff None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:07.008320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6677f-ac26-4e71-9834-3b445e4611ff None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:07.008684 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec6677f-ac26-4e71-9834-3b445e4611ff None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:07.009765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6677f-ac26-4e71-9834-3b445e4611ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:07.012300 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec6677f-ac26-4e71-9834-3b445e4611ff None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:07.030752 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d9ef33-dfa7-4b37-9209-da6618a42ad6 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:07.075053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31c0f6-90f1-4170-8139-955cd10d62dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:07.075962 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31c0f6-90f1-4170-8139-955cd10d62dd None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:07.076369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31c0f6-90f1-4170-8139-955cd10d62dd None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:07.076714 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31c0f6-90f1-4170-8139-955cd10d62dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:07.086074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31c0f6-90f1-4170-8139-955cd10d62dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:07.087647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31c0f6-90f1-4170-8139-955cd10d62dd None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:07.111647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-4e31c0f6-90f1-4170-8139-955cd10d62dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:07.111647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:07.111647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:07.111647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:07.111647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:07.111647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:07.111647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:07.111647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:07.111647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:07.111647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:07.111647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:07.111647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:07.111647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:31:07.111647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:31:07.111647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:31:07.113504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:31:07.113504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:07.113504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:07.113504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:07.113504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:07.113504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:07.113504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:07.113504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:07.113504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:07.113504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:07.113504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:07.113504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:07.113504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:07.113504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:07.113504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:07.113504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:07.113504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:07.115538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:31:07.115538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:31:07.115538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:31:07.115538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:31:07.115538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:31:07.115538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:31:07.115538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:07.115538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:07.115538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 356/708] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:07.116537 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab5bae-fa98-4dd0-bce5-770478c9663e None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:07.116537 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab5bae-fa98-4dd0-bce5-770478c9663e None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:07.116834 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab5bae-fa98-4dd0-bce5-770478c9663e None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:07.122578 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 357/709] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:07.123459 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab5bae-fa98-4dd0-bce5-770478c9663e None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:07.142925 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:07.153463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 353/710] 10.222.0.13 () {60 vars in 1132 bytes} [Fri May 27 19:31:06 2022] POST /identity/v3/users => generated 254 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:31:07.154717 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6677f-ac26-4e71-9834-3b445e4611ff None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:07.335200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab5bae-fa98-4dd0-bce5-770478c9663e None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:07.343262 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec6677f-ac26-4e71-9834-3b445e4611ff None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:07.396839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c53d853-4700-4997-9171-7e5cb7d656b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:07.398011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53d853-4700-4997-9171-7e5cb7d656b0 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:07.398474 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53d853-4700-4997-9171-7e5cb7d656b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:07.399045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c53d853-4700-4997-9171-7e5cb7d656b0 None admin] PATH_INFO: `/v3/services/network` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:07.399905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c53d853-4700-4997-9171-7e5cb7d656b0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:07.401785 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c53d853-4700-4997-9171-7e5cb7d656b0 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:07.402601 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b104c5e-6e2b-4625-99d7-b38a60212851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:07.403377 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b104c5e-6e2b-4625-99d7-b38a60212851 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:07.403668 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b104c5e-6e2b-4625-99d7-b38a60212851 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:07.403948 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b104c5e-6e2b-4625-99d7-b38a60212851 None admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:07.404753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b104c5e-6e2b-4625-99d7-b38a60212851 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:07.405378 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b104c5e-6e2b-4625-99d7-b38a60212851 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:07.406863 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b104c5e-6e2b-4625-99d7-b38a60212851 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:07.419297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-7c53d853-4700-4997-9171-7e5cb7d656b0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 19:31:07.419297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:07.419297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:31:07.419297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:31:07.419297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:31:07.419297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:31:07.419297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:31:07.419297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:07.419297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 19:31:07.419297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:07.419297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:07.419297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:07.419297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:07.419297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:07.419297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:07.419297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:07.419297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:07.420439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:07.420439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:07.420439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:07.420439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:07.420439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:07.420439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:07.420439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:31:07.420439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:31:07.420439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:31:07.420439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:31:07.420439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:07.420439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:07.420439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:07.420439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:07.420439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:07.420439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:07.421335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:07.421335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:07.421335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:07.421335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:07.421335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:07.421335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:07.421335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:07.421335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:07.421335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:07.421335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:07.421335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:31:07.421335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:07.421335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 19:31:07.421335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:07.424666 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 358/711] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:07 2022] GET /identity/v3/services/network => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:07.427232 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbaeef-5369-420d-af49-af948e7a512e None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:07.427723 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbaeef-5369-420d-af49-af948e7a512e None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:07.428262 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbaeef-5369-420d-af49-af948e7a512e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:07.429964 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 354/712] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:31:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:07.433339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b104c5e-6e2b-4625-99d7-b38a60212851 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:07.458698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-9bbbaeef-5369-420d-af49-af948e7a512e None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:07.460296 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-9bbbaeef-5369-420d-af49-af948e7a512e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:07.502678 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:07.576274 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 359/713] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:31:07.578022 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbaeef-5369-420d-af49-af948e7a512e None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:07.590361 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:07.637053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b104c5e-6e2b-4625-99d7-b38a60212851 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:07.707589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50370f8-fce5-48fd-8123-e632ce1a16c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:07.708744 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50370f8-fce5-48fd-8123-e632ce1a16c2 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:07.709435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50370f8-fce5-48fd-8123-e632ce1a16c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:07.711166 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50370f8-fce5-48fd-8123-e632ce1a16c2 None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:07.711166 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50370f8-fce5-48fd-8123-e632ce1a16c2 None admin] RBAC: Adding query filter params (name=network) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:07.711166 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50370f8-fce5-48fd-8123-e632ce1a16c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:07.716980 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50370f8-fce5-48fd-8123-e632ce1a16c2 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:07.720808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 355/714] 10.222.0.13 () {56 vars in 1112 bytes} [Fri May 27 19:31:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:07.721886 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50370f8-fce5-48fd-8123-e632ce1a16c2 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:07.763293 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbaeef-5369-420d-af49-af948e7a512e None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:07.827292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32739e8f-cd84-4852-8128-f04bd5c99eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:07.827292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32739e8f-cd84-4852-8128-f04bd5c99eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:07.827292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32739e8f-cd84-4852-8128-f04bd5c99eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:07.827900 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32739e8f-cd84-4852-8128-f04bd5c99eb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:07.828850 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32739e8f-cd84-4852-8128-f04bd5c99eb2 None admin] RBAC: Adding query filter params (role.id=6505bbb129404436966ee59aa7a01d8e, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=7542e1fd160e4b2c9d25accf19aad19f) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:07.829289 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32739e8f-cd84-4852-8128-f04bd5c99eb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:07.830844 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32739e8f-cd84-4852-8128-f04bd5c99eb2 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:07.853063 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 360/715] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:31:07 2022] GET /identity/v3/role_assignments?user.id=7542e1fd160e4b2c9d25accf19aad19f&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=6505bbb129404436966ee59aa7a01d8e => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:07.853063 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32739e8f-cd84-4852-8128-f04bd5c99eb2 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:07.895478 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50370f8-fce5-48fd-8123-e632ce1a16c2 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:07.934874 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:08.008143 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a29a63-e115-4065-93ed-676ecf356644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:08.009731 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a29a63-e115-4065-93ed-676ecf356644 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:08.010162 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a29a63-e115-4065-93ed-676ecf356644 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:08.011360 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a29a63-e115-4065-93ed-676ecf356644 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:08.012206 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a29a63-e115-4065-93ed-676ecf356644 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:08.015277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a29a63-e115-4065-93ed-676ecf356644 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:08.043231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-a7a29a63-e115-4065-93ed-676ecf356644 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 19:31:08.043231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:08.043231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:31:08.043231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:31:08.043231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:31:08.043231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:31:08.043231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:31:08.043231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:08.043231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 19:31:08.043231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:08.043231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:08.043231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:08.043231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:08.043231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:08.043231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:08.043231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:08.044504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:08.044504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:08.044504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:08.044504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:08.044504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:08.044504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:08.044504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:08.044504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:31:08.044504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:31:08.044504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:31:08.044504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:31:08.044504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:08.044504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:08.044504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:08.044504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:08.044504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:08.045468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:08.045468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:08.045468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:08.045468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:08.045468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:08.045468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:08.045468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:08.045468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:08.045468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:08.045468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:08.045468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:08.045468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:31:08.045468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:08.045468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 19:31:08.045468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:08.053369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:08.061004 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 356/716] 10.222.0.13 () {56 vars in 1129 bytes} [Fri May 27 19:31:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:08.067568 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a29a63-e115-4065-93ed-676ecf356644 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:08.126061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32739e8f-cd84-4852-8128-f04bd5c99eb2 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:08.186920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9b18d-ffaa-40a8-b46b-b2afec4cbfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:08.188129 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9b18d-ffaa-40a8-b46b-b2afec4cbfcd None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:08.188129 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9b18d-ffaa-40a8-b46b-b2afec4cbfcd None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:08.188333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9b18d-ffaa-40a8-b46b-b2afec4cbfcd None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:08.188998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f9b18d-ffaa-40a8-b46b-b2afec4cbfcd None admin] RBAC: Adding query filter params (type=network) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:08.190705 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f9b18d-ffaa-40a8-b46b-b2afec4cbfcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:08.195930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f9b18d-ffaa-40a8-b46b-b2afec4cbfcd None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:08.201717 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 361/717] 10.222.0.13 () {56 vars in 1112 bytes} [Fri May 27 19:31:07 2022] GET /identity/v3/services?type=network => generated 124 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:08.254766 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a29a63-e115-4065-93ed-676ecf356644 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:08.367279 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95a8d3-7c2e-4b88-9e3d-16d2d296a7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:08.367991 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95a8d3-7c2e-4b88-9e3d-16d2d296a7c0 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:08.368338 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95a8d3-7c2e-4b88-9e3d-16d2d296a7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:08.368591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95a8d3-7c2e-4b88-9e3d-16d2d296a7c0 None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:08.369333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95a8d3-7c2e-4b88-9e3d-16d2d296a7c0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:08.369649 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95a8d3-7c2e-4b88-9e3d-16d2d296a7c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:08.371132 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95a8d3-7c2e-4b88-9e3d-16d2d296a7c0 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:08.382782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 357/718] 10.222.0.13 () {56 vars in 1124 bytes} [Fri May 27 19:31:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:08.389451 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9b18d-ffaa-40a8-b46b-b2afec4cbfcd None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:08.582190 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9b18d-ffaa-40a8-b46b-b2afec4cbfcd None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:08.626177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e768a95-8ee4-45ca-bea5-01fd15db3a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:08.626885 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e768a95-8ee4-45ca-bea5-01fd15db3a8c None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:08.627217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e768a95-8ee4-45ca-bea5-01fd15db3a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:08.627502 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e768a95-8ee4-45ca-bea5-01fd15db3a8c None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:08.628254 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e768a95-8ee4-45ca-bea5-01fd15db3a8c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:08.629210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e768a95-8ee4-45ca-bea5-01fd15db3a8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:08.631149 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e768a95-8ee4-45ca-bea5-01fd15db3a8c None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:08.632672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:08.637876 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 362/719] 10.222.0.13 () {56 vars in 1124 bytes} [Fri May 27 19:31:08 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:08.974483 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85d8b1-1215-4546-b780-590087595d47 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:08.974885 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85d8b1-1215-4546-b780-590087595d47 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:08.975111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85d8b1-1215-4546-b780-590087595d47 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:08.976882 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 358/720] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:08.983386 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f5d2b-1402-44bc-9630-2ac6fc7d2ae7 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:08.983721 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f5d2b-1402-44bc-9630-2ac6fc7d2ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:08.984108 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f5d2b-1402-44bc-9630-2ac6fc7d2ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:09.018137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-7f7f5d2b-1402-44bc-9630-2ac6fc7d2ae7 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:09.020207 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-7f7f5d2b-1402-44bc-9630-2ac6fc7d2ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:09.120436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 363/721] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:31:09.148470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85d8b1-1215-4546-b780-590087595d47 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:09.210095 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:09.282937 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85d8b1-1215-4546-b780-590087595d47 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:09.324632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251aef7-d8fd-4c94-b999-218dda55f265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:09.325483 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251aef7-d8fd-4c94-b999-218dda55f265 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:09.325809 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251aef7-d8fd-4c94-b999-218dda55f265 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:09.326101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3251aef7-d8fd-4c94-b999-218dda55f265 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:09.346814 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3251aef7-d8fd-4c94-b999-218dda55f265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:31:09.369111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-3251aef7-d8fd-4c94-b999-218dda55f265 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:31:09.369111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:09.369111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:09.369111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:09.369111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:09.369111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:09.369111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:09.369111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:09.369111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:09.369111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:09.369111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:09.369111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:09.369111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:31:09.369111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:31:09.369111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:31:09.369111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:31:09.370960 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:31:09.370960 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:31:09.370960 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:31:09.370960 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:31:09.370960 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:09.370960 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:09.370960 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:09.370960 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:09.370960 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:09.370960 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:09.370960 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:09.370960 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:09.370960 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:09.370960 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:09.370960 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:09.370960 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:09.372623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:09.372623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:09.372623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:09.372623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:09.372623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:31:09.372623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:31:09.372623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:31:09.372623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:31:09.372623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:31:09.372623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:31:09.372623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:31:09.372623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:09.377472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 359/722] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:31:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:09.384889 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f5d2b-1402-44bc-9630-2ac6fc7d2ae7 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:09.579199 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f5d2b-1402-44bc-9630-2ac6fc7d2ae7 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:09.649586 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:09.670361 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8648f-a3d2-47c8-bbe5-91b31cb0ee39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:09.671050 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8648f-a3d2-47c8-bbe5-91b31cb0ee39 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:09.679137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8648f-a3d2-47c8-bbe5-91b31cb0ee39 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:09.679137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da8648f-a3d2-47c8-bbe5-91b31cb0ee39 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:09.679137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8648f-a3d2-47c8-bbe5-91b31cb0ee39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:09.679137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8648f-a3d2-47c8-bbe5-91b31cb0ee39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:09.679137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da8648f-a3d2-47c8-bbe5-91b31cb0ee39 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:09.692729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 364/723] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:31:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:09.704929 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251aef7-d8fd-4c94-b999-218dda55f265 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:10.001326 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:10.014964 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3251aef7-d8fd-4c94-b999-218dda55f265 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:10.092851 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c10e2e-6998-47eb-8f2b-d2105f2877e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:10.094076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c10e2e-6998-47eb-8f2b-d2105f2877e7 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:10.094734 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c10e2e-6998-47eb-8f2b-d2105f2877e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:10.095230 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c10e2e-6998-47eb-8f2b-d2105f2877e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:10.126277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c10e2e-6998-47eb-8f2b-d2105f2877e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:10.128128 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c10e2e-6998-47eb-8f2b-d2105f2877e7 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:10.156367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-75c10e2e-6998-47eb-8f2b-d2105f2877e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:10.156367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:10.156367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:10.156367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:10.156367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:10.156367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:10.156367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:10.156367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:10.156367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:10.156367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:10.156367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:10.156367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:10.156367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:10.156367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:10.156367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:10.157857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:10.157857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:10.157857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:10.157857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:10.157857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:10.157857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:10.157857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:10.157857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:10.157857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:10.157857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:10.157857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:10.157857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:10.157857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:10.157857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:10.157857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:10.157857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:10.157857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:10.158756 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:10.158756 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:10.158756 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:10.158756 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:10.167153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 360/724] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 463 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:10.172603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8648f-a3d2-47c8-bbe5-91b31cb0ee39 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:10.391655 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da8648f-a3d2-47c8-bbe5-91b31cb0ee39 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:10.466952 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70db9a4-2fbb-4f7c-8719-317c8518d522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:10.467674 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70db9a4-2fbb-4f7c-8719-317c8518d522 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:10.467970 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70db9a4-2fbb-4f7c-8719-317c8518d522 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:10.468255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70db9a4-2fbb-4f7c-8719-317c8518d522 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:10.469064 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70db9a4-2fbb-4f7c-8719-317c8518d522 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:10.469381 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70db9a4-2fbb-4f7c-8719-317c8518d522 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:10.470840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70db9a4-2fbb-4f7c-8719-317c8518d522 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:10.502174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 365/725] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:10.512768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c10e2e-6998-47eb-8f2b-d2105f2877e7 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:10.560352 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd870903-2f1c-416b-ace3-48061de17eef None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:10.561059 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd870903-2f1c-416b-ace3-48061de17eef None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:10.563569 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd870903-2f1c-416b-ace3-48061de17eef None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:10.565508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 366/726] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:10.572886 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78058d6f-902c-4cb5-948c-6a2d1eb3bd1b None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:10.573513 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78058d6f-902c-4cb5-948c-6a2d1eb3bd1b None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:10.576400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78058d6f-902c-4cb5-948c-6a2d1eb3bd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:10.621109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-78058d6f-902c-4cb5-948c-6a2d1eb3bd1b None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:10.622676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-78058d6f-902c-4cb5-948c-6a2d1eb3bd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:10.727386 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:10.745766 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c10e2e-6998-47eb-8f2b-d2105f2877e7 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:10.775932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 367/727] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:31:10.810347 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a04438-0a64-4dbc-bde3-98634efc9b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:10.811527 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a04438-0a64-4dbc-bde3-98634efc9b47 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:10.812017 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a04438-0a64-4dbc-bde3-98634efc9b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:10.812711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a04438-0a64-4dbc-bde3-98634efc9b47 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:10.822618 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a04438-0a64-4dbc-bde3-98634efc9b47 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:10.824630 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78058d6f-902c-4cb5-948c-6a2d1eb3bd1b None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:10.837433 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a04438-0a64-4dbc-bde3-98634efc9b47 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:10.871643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-94a04438-0a64-4dbc-bde3-98634efc9b47 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 19:31:10.871643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:10.871643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:10.871643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:10.871643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:10.871643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:10.871643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:10.871643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:10.871643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:10.871643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:10.871643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:10.871643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:10.871643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:31:10.871643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:31:10.871643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:31:10.871643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:31:10.872692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:10.872692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:10.872692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:31:10.872692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:10.872692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:31:10.872692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:10.872692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:10.872692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:10.872692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:10.872692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:10.872692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:10.872692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:10.872692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:10.872692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:10.872692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:10.872692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:10.872692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:10.872692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:10.873675 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:10.873675 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:10.873675 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:31:10.873675 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:31:10.873675 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:31:10.873675 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:31:10.873675 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:31:10.873675 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:31:10.873675 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 19:31:10.873675 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:10.883023 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 361/728] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:31:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:10.901889 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a04438-0a64-4dbc-bde3-98634efc9b47 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:10.921357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:11.059766 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78058d6f-902c-4cb5-948c-6a2d1eb3bd1b None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:11.141370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a04438-0a64-4dbc-bde3-98634efc9b47 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:11.150474 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe23a79-ce7d-491b-8e8c-9bcaefb47c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:11.151306 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe23a79-ce7d-491b-8e8c-9bcaefb47c81 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:11.151590 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe23a79-ce7d-491b-8e8c-9bcaefb47c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:11.151868 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe23a79-ce7d-491b-8e8c-9bcaefb47c81 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:11.185699 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe23a79-ce7d-491b-8e8c-9bcaefb47c81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:31:11.207490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac07bb-b37f-4296-9bc2-35db66965ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:11.208205 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac07bb-b37f-4296-9bc2-35db66965ddf None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:11.208616 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac07bb-b37f-4296-9bc2-35db66965ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:11.208901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac07bb-b37f-4296-9bc2-35db66965ddf None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:11.209836 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac07bb-b37f-4296-9bc2-35db66965ddf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:11.210153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac07bb-b37f-4296-9bc2-35db66965ddf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:11.211900 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac07bb-b37f-4296-9bc2-35db66965ddf None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:11.213461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-cbe23a79-ce7d-491b-8e8c-9bcaefb47c81 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 19:31:11.213461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:11.213461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:11.213461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:11.213461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:11.213461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:11.213461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:11.213461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:11.213461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:11.213461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:11.213461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:11.213461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:11.213461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:31:11.213461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:31:11.213461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:31:11.213461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:31:11.214380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:31:11.214380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:31:11.214380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:31:11.214380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:31:11.214380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:11.214380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:11.214380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:11.214380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:11.214380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:11.214380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:11.214380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:11.214380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:11.214380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:11.214380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:11.214380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:11.214380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:11.215228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:11.215228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:11.215228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:11.215228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:11.215228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:31:11.215228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:31:11.215228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:31:11.215228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:31:11.215228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:31:11.215228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:31:11.215228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 19:31:11.215228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:11.215228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 368/729] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:31:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:31:11.216586 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736246b-0be9-4892-9868-bd08472bd65f None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:11.216878 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736246b-0be9-4892-9868-bd08472bd65f None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:11.217211 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736246b-0be9-4892-9868-bd08472bd65f None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:11.223982 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 369/730] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:11.225448 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9763e-a9e1-4384-b0e4-1f128896cfc0 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:11.225788 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9763e-a9e1-4384-b0e4-1f128896cfc0 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:11.226076 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9763e-a9e1-4384-b0e4-1f128896cfc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:11.255089 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 362/731] 10.222.0.13 () {56 vars in 1135 bytes} [Fri May 27 19:31:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:11.256007 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac07bb-b37f-4296-9bc2-35db66965ddf None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:11.263201 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-deb9763e-a9e1-4384-b0e4-1f128896cfc0 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:11.271853 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-deb9763e-a9e1-4384-b0e4-1f128896cfc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:11.357784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:11.386862 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:11.404363 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 370/732] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:31:11.405485 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9763e-a9e1-4384-b0e4-1f128896cfc0 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:11.429379 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac07bb-b37f-4296-9bc2-35db66965ddf None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:11.472431 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98950b-fcc4-4fbd-8a76-33251057fb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:11.473532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98950b-fcc4-4fbd-8a76-33251057fb5c None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:11.473996 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98950b-fcc4-4fbd-8a76-33251057fb5c None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:11.474421 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98950b-fcc4-4fbd-8a76-33251057fb5c None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:11.475377 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98950b-fcc4-4fbd-8a76-33251057fb5c None admin] RBAC: Adding query filter params (name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:11.475695 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98950b-fcc4-4fbd-8a76-33251057fb5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:11.477128 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98950b-fcc4-4fbd-8a76-33251057fb5c None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:11.487650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 363/733] 10.222.0.13 () {56 vars in 1103 bytes} [Fri May 27 19:31:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:11.489370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9669ff4-b592-4519-9c16-36a8cca31352 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:11.489801 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9669ff4-b592-4519-9c16-36a8cca31352 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:11.490213 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9669ff4-b592-4519-9c16-36a8cca31352 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:11.491935 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 364/734] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:11.493099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9669ff4-b592-4519-9c16-36a8cca31352 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:11.529786 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9763e-a9e1-4384-b0e4-1f128896cfc0 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:11.577846 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b4eef-5a4f-47e1-9b0c-13f16f58572c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:11.579053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b4eef-5a4f-47e1-9b0c-13f16f58572c None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:11.579523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b4eef-5a4f-47e1-9b0c-13f16f58572c None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:11.580349 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b4eef-5a4f-47e1-9b0c-13f16f58572c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:11.633887 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:11.639774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9669ff4-b592-4519-9c16-36a8cca31352 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:11.686985 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:11.692439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b4eef-5a4f-47e1-9b0c-13f16f58572c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:11.695005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b4eef-5a4f-47e1-9b0c-13f16f58572c None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:11.704499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab887f9a-31b6-4bb4-8818-573e10e1bb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:11.706798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab887f9a-31b6-4bb4-8818-573e10e1bb45 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:11.708497 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab887f9a-31b6-4bb4-8818-573e10e1bb45 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:11.709050 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab887f9a-31b6-4bb4-8818-573e10e1bb45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:11.722724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab887f9a-31b6-4bb4-8818-573e10e1bb45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:31:11.731439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-ab887f9a-31b6-4bb4-8818-573e10e1bb45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:31:11.731439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:11.731439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:11.731439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:11.731439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:11.731439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:11.731439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:11.731439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:11.731439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:11.731439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:11.731439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:11.731439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:11.731439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:31:11.731439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:31:11.731439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:31:11.731439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:31:11.732451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:31:11.732451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:31:11.732451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:31:11.732451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:31:11.732451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:11.732451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:11.732451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:11.732451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:11.732451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:11.732451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:11.732451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:11.732451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:11.732451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:11.732451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:11.732451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:11.732451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:11.733909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:11.733909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:11.733909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:11.733909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:11.733909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:31:11.733909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:31:11.733909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:31:11.733909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:31:11.733909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:31:11.733909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:31:11.733909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:31:11.733909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:11.736449 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 365/735] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:31:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:11.737811 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab887f9a-31b6-4bb4-8818-573e10e1bb45 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:11.863877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 371/736] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:31:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 19:31:11.865359 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ea8ad-42f5-4d27-bfb0-4e81ab7d6065 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:11.865868 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ea8ad-42f5-4d27-bfb0-4e81ab7d6065 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:11.866413 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ea8ad-42f5-4d27-bfb0-4e81ab7d6065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:11.881841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab887f9a-31b6-4bb4-8818-573e10e1bb45 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:11.886870 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-c24ea8ad-42f5-4d27-bfb0-4e81ab7d6065 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:11.888049 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-c24ea8ad-42f5-4d27-bfb0-4e81ab7d6065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:11.923265 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ba390-5ea9-413c-94a3-0e1a6b4624b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:11.924066 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ba390-5ea9-413c-94a3-0e1a6b4624b9 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:11.924348 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ba390-5ea9-413c-94a3-0e1a6b4624b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:11.925150 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ba390-5ea9-413c-94a3-0e1a6b4624b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:11.936778 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ba390-5ea9-413c-94a3-0e1a6b4624b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:11.938299 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ba390-5ea9-413c-94a3-0e1a6b4624b9 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:11.958460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-413ba390-5ea9-413c-94a3-0e1a6b4624b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:11.958460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:11.958460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:11.958460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:11.958460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:11.958460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:11.958460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:11.958460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:11.958460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:11.958460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:11.958460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:11.958460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:11.958460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:11.958460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:11.958460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:11.959450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:11.959450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:11.959450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:11.959450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:11.959450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:11.959450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:11.959450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:11.959450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:11.959450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:11.959450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:11.959450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:11.959450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:11.959450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:11.959450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:11.959450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:11.959450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:11.959450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:11.960394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:11.960394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:11.960394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:11.960394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:11.960394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 366/737] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:11.961273 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ba390-5ea9-413c-94a3-0e1a6b4624b9 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:11.961376 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:11.969059 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 372/738] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:31:11.978660 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1b32a-bdb6-4bb3-b089-b0df3bc0a2f4 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:11.978660 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1b32a-bdb6-4bb3-b089-b0df3bc0a2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:11.978660 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1b32a-bdb6-4bb3-b089-b0df3bc0a2f4 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:11.978660 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 373/739] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:11.978660 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1b32a-bdb6-4bb3-b089-b0df3bc0a2f4 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:12.020160 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:12.089645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ba390-5ea9-413c-94a3-0e1a6b4624b9 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:12.106391 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1b32a-bdb6-4bb3-b089-b0df3bc0a2f4 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:12.135275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cd5a9-60dd-4dc1-923a-4a0c7123b422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:12.136054 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cd5a9-60dd-4dc1-923a-4a0c7123b422 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:12.136416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cd5a9-60dd-4dc1-923a-4a0c7123b422 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:12.137034 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44cd5a9-60dd-4dc1-923a-4a0c7123b422 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:12.138106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44cd5a9-60dd-4dc1-923a-4a0c7123b422 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:12.138576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44cd5a9-60dd-4dc1-923a-4a0c7123b422 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:12.140682 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44cd5a9-60dd-4dc1-923a-4a0c7123b422 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:12.152337 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 367/740] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:31:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:12.153261 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cd5a9-60dd-4dc1-923a-4a0c7123b422 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:12.155877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2bc726-526a-4e48-921d-69ad8f124e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:12.157199 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2bc726-526a-4e48-921d-69ad8f124e37 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:12.157679 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2bc726-526a-4e48-921d-69ad8f124e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:12.157953 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2bc726-526a-4e48-921d-69ad8f124e37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:12.169826 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2bc726-526a-4e48-921d-69ad8f124e37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:12.171305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2bc726-526a-4e48-921d-69ad8f124e37 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:12.192810 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-3a2bc726-526a-4e48-921d-69ad8f124e37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:12.192810 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:12.192810 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:12.192810 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:12.192810 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:12.192810 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:12.192810 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:12.192810 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:12.192810 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:12.192810 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:12.192810 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:12.192810 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:12.192810 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:12.192810 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:12.192810 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:12.193941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:12.193941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:12.193941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:12.193941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:12.193941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:12.193941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:12.193941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:12.193941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:12.193941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:12.193941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:12.193941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:12.193941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:12.193941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:12.193941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:12.193941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:12.193941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:12.193941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:12.195353 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:12.195353 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:12.195353 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:12.195353 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:12.195353 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 374/741] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:12.195353 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b211511-d5b2-428d-8d0a-95d27e27d012 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:12.195762 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b211511-d5b2-428d-8d0a-95d27e27d012 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:12.196048 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b211511-d5b2-428d-8d0a-95d27e27d012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:12.213533 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-2b211511-d5b2-428d-8d0a-95d27e27d012 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:12.214730 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-2b211511-d5b2-428d-8d0a-95d27e27d012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:12.275566 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:12.277930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44cd5a9-60dd-4dc1-923a-4a0c7123b422 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:12.280648 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:12.297691 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 375/742] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 19:31:12.298992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b211511-d5b2-428d-8d0a-95d27e27d012 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:12.318327 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35c7fe-18e7-4c48-b0f2-2676d886fa3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:12.319042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35c7fe-18e7-4c48-b0f2-2676d886fa3b None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:12.319325 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35c7fe-18e7-4c48-b0f2-2676d886fa3b None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:12.319595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35c7fe-18e7-4c48-b0f2-2676d886fa3b None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:12.320333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35c7fe-18e7-4c48-b0f2-2676d886fa3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:12.320640 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35c7fe-18e7-4c48-b0f2-2676d886fa3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:12.322529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35c7fe-18e7-4c48-b0f2-2676d886fa3b None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:12.336773 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 368/743] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:12.337765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35c7fe-18e7-4c48-b0f2-2676d886fa3b None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:12.443316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b211511-d5b2-428d-8d0a-95d27e27d012 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:12.483947 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35c7fe-18e7-4c48-b0f2-2676d886fa3b None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:12.511906 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92fc2ec-1e99-498b-acf9-3892c46af0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:12.512107 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fc2ec-1e99-498b-acf9-3892c46af0c5 None admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:12.512387 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fc2ec-1e99-498b-acf9-3892c46af0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:12.512669 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92fc2ec-1e99-498b-acf9-3892c46af0c5 None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:12.513446 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92fc2ec-1e99-498b-acf9-3892c46af0c5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:12.514857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92fc2ec-1e99-498b-acf9-3892c46af0c5 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:12.537596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea7d01-1bd7-49f3-ba64-857ec9b1d24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:12.538425 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea7d01-1bd7-49f3-ba64-857ec9b1d24c None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:12.538848 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea7d01-1bd7-49f3-ba64-857ec9b1d24c None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:12.539327 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea7d01-1bd7-49f3-ba64-857ec9b1d24c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:12.557665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea7d01-1bd7-49f3-ba64-857ec9b1d24c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:12.559717 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea7d01-1bd7-49f3-ba64-857ec9b1d24c None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:12.586619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-90ea7d01-1bd7-49f3-ba64-857ec9b1d24c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:12.586619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:12.586619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:12.586619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:12.586619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:12.586619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:12.586619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:12.586619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:12.586619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:12.586619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:12.586619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:12.586619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:12.586619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:12.586619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:12.586619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:12.587753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:12.587753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:12.587753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:12.587753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:12.587753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:12.587753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:12.587753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:12.587753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:12.587753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:12.587753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:12.587753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:12.587753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:12.587753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:12.587753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:12.587753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:12.587753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:12.587753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:12.588680 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:12.588680 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:12.588680 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:12.588680 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:12.588680 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 369/744] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:12.589171 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea7d01-1bd7-49f3-ba64-857ec9b1d24c None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:12.610818 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:12.645021 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 376/745] 10.222.0.13 () {60 vars in 1141 bytes} [Fri May 27 19:31:12 2022] POST /identity/v3/services => generated 240 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:31:12.645021 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92fc2ec-1e99-498b-acf9-3892c46af0c5 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:12.710901 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:12.732526 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea7d01-1bd7-49f3-ba64-857ec9b1d24c None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:12.797805 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a052f-0981-43b0-8e83-7c697be49ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:12.798625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a052f-0981-43b0-8e83-7c697be49ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:12.798977 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a052f-0981-43b0-8e83-7c697be49ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:12.799332 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03a052f-0981-43b0-8e83-7c697be49ce4 None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:12.800595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a052f-0981-43b0-8e83-7c697be49ce4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:12.801047 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a052f-0981-43b0-8e83-7c697be49ce4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:12.803707 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03a052f-0981-43b0-8e83-7c697be49ce4 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:12.817969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92fc2ec-1e99-498b-acf9-3892c46af0c5 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:12.822536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 370/746] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:12.822536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a052f-0981-43b0-8e83-7c697be49ce4 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:12.875312 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dfd7f9-3828-4a5e-a909-231ef17e0aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:12.879294 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfd7f9-3828-4a5e-a909-231ef17e0aab None admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:12.879294 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfd7f9-3828-4a5e-a909-231ef17e0aab None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:12.879294 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfd7f9-3828-4a5e-a909-231ef17e0aab None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:12.879294 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dfd7f9-3828-4a5e-a909-231ef17e0aab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:12.879294 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dfd7f9-3828-4a5e-a909-231ef17e0aab None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:12.911695 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:12.946437 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03a052f-0981-43b0-8e83-7c697be49ce4 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:12.947703 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 377/747] 10.222.0.13 () {60 vars in 1141 bytes} [Fri May 27 19:31:12 2022] POST /identity/v3/services => generated 251 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:31:12.948323 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dfd7f9-3828-4a5e-a909-231ef17e0aab None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:13.029642 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb59994-fb20-49ef-b39c-542ddb287c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:13.030891 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb59994-fb20-49ef-b39c-542ddb287c9d None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:13.031418 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb59994-fb20-49ef-b39c-542ddb287c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:13.031917 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb59994-fb20-49ef-b39c-542ddb287c9d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:13.052817 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb59994-fb20-49ef-b39c-542ddb287c9d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:13.056609 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb59994-fb20-49ef-b39c-542ddb287c9d None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:13.094584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-deb59994-fb20-49ef-b39c-542ddb287c9d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 19:31:13.094584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:13.094584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:13.094584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:13.094584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:13.094584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:13.094584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:13.094584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:13.094584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:13.094584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:13.094584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:13.094584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:13.094584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:31:13.094584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:31:13.094584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:31:13.094584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:31:13.095596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:13.095596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:13.095596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:31:13.095596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:13.095596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:31:13.095596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:13.095596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:13.095596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:13.095596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:13.095596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:13.095596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:13.095596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:13.095596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:13.095596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:13.095596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:13.095596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:13.095596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:13.095596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:13.096595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:13.096595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:13.096595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:31:13.096595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:31:13.096595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:31:13.096595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:31:13.096595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:31:13.096595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:31:13.096595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 19:31:13.096595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:13.111190 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 371/748] 10.222.0.13 () {56 vars in 1117 bytes} [Fri May 27 19:31:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:13.111190 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb59994-fb20-49ef-b39c-542ddb287c9d None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:13.153534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:13.158550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dfd7f9-3828-4a5e-a909-231ef17e0aab None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:13.216213 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea6da2-df79-4c6e-a300-e593abd8fa8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:13.216896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea6da2-df79-4c6e-a300-e593abd8fa8c None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:13.217216 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea6da2-df79-4c6e-a300-e593abd8fa8c None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:13.217503 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea6da2-df79-4c6e-a300-e593abd8fa8c None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:13.218273 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea6da2-df79-4c6e-a300-e593abd8fa8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:13.218607 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea6da2-df79-4c6e-a300-e593abd8fa8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:13.220079 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ea6da2-df79-4c6e-a300-e593abd8fa8c None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:13.238672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 378/749] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:13.239039 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea6da2-df79-4c6e-a300-e593abd8fa8c None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:13.304394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb59994-fb20-49ef-b39c-542ddb287c9d None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:13.337421 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:13.346835 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bddcb-b97a-4d5a-b6f5-567096fbc5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:13.347632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bddcb-b97a-4d5a-b6f5-567096fbc5ed None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:13.348047 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bddcb-b97a-4d5a-b6f5-567096fbc5ed None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:13.348333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bddcb-b97a-4d5a-b6f5-567096fbc5ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:13.358510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84bddcb-b97a-4d5a-b6f5-567096fbc5ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:13.362864 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84bddcb-b97a-4d5a-b6f5-567096fbc5ed None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:13.380670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-f84bddcb-b97a-4d5a-b6f5-567096fbc5ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:13.380670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:13.380670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:13.380670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:13.380670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:13.380670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:13.380670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:13.380670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:13.380670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:13.380670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:13.380670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:13.380670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:13.380670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:31:13.380670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:31:13.380670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:31:13.381738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:31:13.381738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:13.381738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:13.381738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:13.381738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:13.381738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:13.381738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:13.381738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:13.381738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:13.381738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:13.381738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:13.381738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:13.381738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:13.381738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:13.381738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:13.381738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:13.381738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:13.382653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:31:13.382653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:31:13.382653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:31:13.382653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:31:13.382653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:31:13.382653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:31:13.382653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:13.382653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:13.383610 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 372/750] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:13.385280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bddcb-b97a-4d5a-b6f5-567096fbc5ed None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:13.452008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ea6da2-df79-4c6e-a300-e593abd8fa8c None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:13.530718 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bddcb-b97a-4d5a-b6f5-567096fbc5ed None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:13.557074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7f6c7-c01c-4967-b163-f6781bb0498f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:13.558107 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7f6c7-c01c-4967-b163-f6781bb0498f None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:13.558602 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7f6c7-c01c-4967-b163-f6781bb0498f None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:13.559087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7f6c7-c01c-4967-b163-f6781bb0498f None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:13.560229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7f6c7-c01c-4967-b163-f6781bb0498f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:13.560752 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7f6c7-c01c-4967-b163-f6781bb0498f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:13.565322 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7f6c7-c01c-4967-b163-f6781bb0498f None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:13.573840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333051da-0286-4933-8acc-e588279419f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:13.575199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333051da-0286-4933-8acc-e588279419f8 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:13.575645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333051da-0286-4933-8acc-e588279419f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:13.576104 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333051da-0286-4933-8acc-e588279419f8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:13.586831 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333051da-0286-4933-8acc-e588279419f8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:13.588307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333051da-0286-4933-8acc-e588279419f8 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:13.607916 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 379/751] 10.222.0.13 () {56 vars in 1149 bytes} [Fri May 27 19:31:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:13.608234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-333051da-0286-4933-8acc-e588279419f8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 19:31:13.608234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:13.608234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:13.608234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:13.608234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:13.608234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:13.608234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:13.608234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:13.608234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:13.608234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:13.608234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:13.608234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:13.608234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:31:13.608234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:31:13.608234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:31:13.608234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:31:13.609251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:13.609251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:13.609251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:31:13.609251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:13.609251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:31:13.609251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:13.609251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:13.609251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:13.609251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:13.609251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:13.609251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:13.609251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:13.609251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:13.609251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:13.609251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:13.609251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:13.609251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:13.609251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:13.610223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:13.610223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:13.610223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:31:13.610223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:31:13.610223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:31:13.610223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:31:13.610223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:31:13.610223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:31:13.610223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 19:31:13.610223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:13.610223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 373/752] 10.222.0.13 () {56 vars in 1093 bytes} [Fri May 27 19:31:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:13.610820 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333051da-0286-4933-8acc-e588279419f8 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:13.619018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7f6c7-c01c-4967-b163-f6781bb0498f None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:13.648060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:13.683923 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:13.743231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333051da-0286-4933-8acc-e588279419f8 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:13.808092 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeec335-c7e4-4140-a8d2-016c2d083d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:13.808693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeec335-c7e4-4140-a8d2-016c2d083d46 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:13.810277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeec335-c7e4-4140-a8d2-016c2d083d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:13.810590 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfeec335-c7e4-4140-a8d2-016c2d083d46 None admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:13.811664 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeec335-c7e4-4140-a8d2-016c2d083d46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:13.811979 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeec335-c7e4-4140-a8d2-016c2d083d46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:13.814482 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfeec335-c7e4-4140-a8d2-016c2d083d46 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:13.817125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7f6c7-c01c-4967-b163-f6781bb0498f None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:13.833214 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 374/753] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:31:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:13.833604 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeec335-c7e4-4140-a8d2-016c2d083d46 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:13.890584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c892ff-0746-41ee-addb-4cea773e631c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:13.891990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c892ff-0746-41ee-addb-4cea773e631c None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:13.892481 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c892ff-0746-41ee-addb-4cea773e631c None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:13.893123 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c892ff-0746-41ee-addb-4cea773e631c None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:13.894204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c892ff-0746-41ee-addb-4cea773e631c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:13.894785 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c892ff-0746-41ee-addb-4cea773e631c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:13.899592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c892ff-0746-41ee-addb-4cea773e631c None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:13.936438 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 380/754] 10.222.0.13 () {56 vars in 1133 bytes} [Fri May 27 19:31:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:13.939814 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c892ff-0746-41ee-addb-4cea773e631c None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:13.965293 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfeec335-c7e4-4140-a8d2-016c2d083d46 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:14.008533 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb127dd-304e-4dd0-804e-6b1c8eed9b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:14.010761 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb127dd-304e-4dd0-804e-6b1c8eed9b5f None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:14.011167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb127dd-304e-4dd0-804e-6b1c8eed9b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:14.011456 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb127dd-304e-4dd0-804e-6b1c8eed9b5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:14.025321 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:14.077495 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:14.102603 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb127dd-304e-4dd0-804e-6b1c8eed9b5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:14.104877 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb127dd-304e-4dd0-804e-6b1c8eed9b5f None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:14.107982 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c892ff-0746-41ee-addb-4cea773e631c None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:14.181100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce47e3c-162d-41c8-8b44-5848750fee9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:14.181873 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce47e3c-162d-41c8-8b44-5848750fee9e None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:14.182236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce47e3c-162d-41c8-8b44-5848750fee9e None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:14.182585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce47e3c-162d-41c8-8b44-5848750fee9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:14.183502 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce47e3c-162d-41c8-8b44-5848750fee9e None admin] RBAC: Adding query filter params (role.id=6505bbb129404436966ee59aa7a01d8e, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=37bc6d0abb454de19e1ab2f46bf4a593) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:14.183890 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce47e3c-162d-41c8-8b44-5848750fee9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:14.189513 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce47e3c-162d-41c8-8b44-5848750fee9e None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:14.211208 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 381/755] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:31:13 2022] GET /identity/v3/role_assignments?user.id=37bc6d0abb454de19e1ab2f46bf4a593&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=6505bbb129404436966ee59aa7a01d8e => generated 259 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:14.212369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce47e3c-162d-41c8-8b44-5848750fee9e None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:14.234159 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 375/756] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:31:13 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 19:31:14.241445 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb127dd-304e-4dd0-804e-6b1c8eed9b5f None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:14.268178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:14.427409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb127dd-304e-4dd0-804e-6b1c8eed9b5f None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:14.441173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce47e3c-162d-41c8-8b44-5848750fee9e None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:14.519441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28c674-e57b-4022-8bee-4caa5487e7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:14.521720 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28c674-e57b-4022-8bee-4caa5487e7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:14.522012 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28c674-e57b-4022-8bee-4caa5487e7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:14.522296 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28c674-e57b-4022-8bee-4caa5487e7c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:14.528144 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b58e9-e987-44d6-847d-bcf573f8ad04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:14.528686 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b58e9-e987-44d6-847d-bcf573f8ad04 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:14.529057 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b58e9-e987-44d6-847d-bcf573f8ad04 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:14.529348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b58e9-e987-44d6-847d-bcf573f8ad04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:14.546435 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:14.547925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a28c674-e57b-4022-8bee-4caa5487e7c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:14.552016 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a28c674-e57b-4022-8bee-4caa5487e7c2 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:14.556679 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b58e9-e987-44d6-847d-bcf573f8ad04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:14.558279 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b58e9-e987-44d6-847d-bcf573f8ad04 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:14.593415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-3a28c674-e57b-4022-8bee-4caa5487e7c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:14.593415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:14.593415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:14.593415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:14.593415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:14.593415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:14.593415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:14.593415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:14.593415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:14.593415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:14.593415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:14.593415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:14.593415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:14.593415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:14.593415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:14.594397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:14.594397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:14.594397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:14.594397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:14.594397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:14.594397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:14.594397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:14.594397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:14.594397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:14.594397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:14.594397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:14.594397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:14.594397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:14.594397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:14.594397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:14.594397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:14.594397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:14.595357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:14.595357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:14.595357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:14.595357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:14.595357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 376/757] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:31:14.599148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-df6b58e9-e987-44d6-847d-bcf573f8ad04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:14.599148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:14.599148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:14.599148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:14.599148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:14.599148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:14.599148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:14.599148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:14.599148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:14.599148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:14.599148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:14.599148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:14.599148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:14.599148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:14.599148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:14.600033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:14.600033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:14.600033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:14.600033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:14.600033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:14.600033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:14.600033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:14.600033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:14.600033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:14.600033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:14.600033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:14.600033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:14.600033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:14.600033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:14.600033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:14.600033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:14.600033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:14.601255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:14.601255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:14.601255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:14.601255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:14.602716 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28c674-e57b-4022-8bee-4caa5487e7c2 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:14.607184 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 382/758] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:14.618702 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b58e9-e987-44d6-847d-bcf573f8ad04 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:14.729195 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:14.815395 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28c674-e57b-4022-8bee-4caa5487e7c2 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:14.838421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b58e9-e987-44d6-847d-bcf573f8ad04 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:14.896300 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da291eb7-4e86-449b-9e3f-1f75a938ccf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:14.897364 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da291eb7-4e86-449b-9e3f-1f75a938ccf9 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:14.898177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da291eb7-4e86-449b-9e3f-1f75a938ccf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:14.899926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da291eb7-4e86-449b-9e3f-1f75a938ccf9 None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:14.900887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da291eb7-4e86-449b-9e3f-1f75a938ccf9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:14.902201 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da291eb7-4e86-449b-9e3f-1f75a938ccf9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:14.904480 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da291eb7-4e86-449b-9e3f-1f75a938ccf9 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:14.921584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a4659-d77a-47cd-9d59-78654d503fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:14.923002 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a4659-d77a-47cd-9d59-78654d503fca None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:14.923490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a4659-d77a-47cd-9d59-78654d503fca None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:14.923994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a4659-d77a-47cd-9d59-78654d503fca None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:14.925596 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a4659-d77a-47cd-9d59-78654d503fca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:14.926113 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a4659-d77a-47cd-9d59-78654d503fca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:14.928712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a4659-d77a-47cd-9d59-78654d503fca None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:14.935178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 377/759] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:14.941214 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da291eb7-4e86-449b-9e3f-1f75a938ccf9 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:14.959617 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 383/760] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:31:14.965316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a4659-d77a-47cd-9d59-78654d503fca None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:15.109209 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da291eb7-4e86-449b-9e3f-1f75a938ccf9 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:15.131719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:15.149362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:15.150638 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8219b7-8731-477c-b5b2-6232c6b492db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:15.151552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8219b7-8731-477c-b5b2-6232c6b492db None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:15.151940 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8219b7-8731-477c-b5b2-6232c6b492db None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:15.152320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8219b7-8731-477c-b5b2-6232c6b492db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:15.165688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8219b7-8731-477c-b5b2-6232c6b492db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:15.165688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8219b7-8731-477c-b5b2-6232c6b492db None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:15.183942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-3f8219b7-8731-477c-b5b2-6232c6b492db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:15.183942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:15.183942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:15.183942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:15.183942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:15.183942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:15.183942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:15.183942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:15.183942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:15.183942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:15.183942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:15.183942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:15.183942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:31:15.183942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:31:15.183942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:31:15.185930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:31:15.185930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:15.185930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:15.185930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:15.185930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:15.185930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:15.185930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:15.185930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:15.185930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:15.185930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:15.185930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:15.185930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:15.185930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:15.185930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:15.185930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:15.185930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:15.185930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:15.187062 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:31:15.187062 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:31:15.187062 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:31:15.187062 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:31:15.187062 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:31:15.187062 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:31:15.187062 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:15.187062 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:15.191919 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 378/761] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:15.195494 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8219b7-8731-477c-b5b2-6232c6b492db None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:15.249374 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a4659-d77a-47cd-9d59-78654d503fca None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:15.337354 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db5e39-ea02-4f05-bbf9-d6df71ae8bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:15.338270 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db5e39-ea02-4f05-bbf9-d6df71ae8bdc None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:15.338622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db5e39-ea02-4f05-bbf9-d6df71ae8bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:15.338988 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db5e39-ea02-4f05-bbf9-d6df71ae8bdc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:15.348026 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8219b7-8731-477c-b5b2-6232c6b492db None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:15.358254 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db5e39-ea02-4f05-bbf9-d6df71ae8bdc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:15.359898 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db5e39-ea02-4f05-bbf9-d6df71ae8bdc None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:15.379508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-93db5e39-ea02-4f05-bbf9-d6df71ae8bdc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:15.379508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:15.379508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:15.379508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:15.379508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:15.379508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:15.379508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:15.379508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:15.379508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:15.379508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:15.379508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:15.379508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:15.379508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:31:15.379508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:31:15.379508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:31:15.380461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:31:15.380461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:15.380461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:15.380461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:15.380461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:15.380461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:15.380461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:15.380461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:15.380461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:15.380461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:15.380461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:15.380461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:15.380461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:15.380461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:15.380461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:15.380461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:15.380461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:15.381414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:31:15.381414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:31:15.381414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:31:15.381414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:31:15.381414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:31:15.381414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:31:15.381414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:15.381414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:15.384217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 384/762] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 418 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:31:15.394463 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058b022-f17e-428b-9630-9c35e6c06159 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:15.394838 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058b022-f17e-428b-9630-9c35e6c06159 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:15.395123 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058b022-f17e-428b-9630-9c35e6c06159 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:15.396705 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 385/763] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:15 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:15.401327 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058b022-f17e-428b-9630-9c35e6c06159 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:15.420242 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718d8d97-4d35-4ff3-9a1b-9ca340027a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:15.420925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d8d97-4d35-4ff3-9a1b-9ca340027a07 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:15.421248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d8d97-4d35-4ff3-9a1b-9ca340027a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:15.421759 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718d8d97-4d35-4ff3-9a1b-9ca340027a07 None admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:15.422574 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d8d97-4d35-4ff3-9a1b-9ca340027a07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:15.422887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d8d97-4d35-4ff3-9a1b-9ca340027a07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:15.424491 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718d8d97-4d35-4ff3-9a1b-9ca340027a07 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:15.462855 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 379/764] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:31:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:15.464441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51910d74-1645-4f1d-b49c-6da18726b5f4 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:15.464737 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51910d74-1645-4f1d-b49c-6da18726b5f4 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:15.465046 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51910d74-1645-4f1d-b49c-6da18726b5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:15.493963 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-51910d74-1645-4f1d-b49c-6da18726b5f4 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:15.495399 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-51910d74-1645-4f1d-b49c-6da18726b5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:15.575216 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:15.608970 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058b022-f17e-428b-9630-9c35e6c06159 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:15.634396 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:15.641272 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 380/765] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:15.642366 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51910d74-1645-4f1d-b49c-6da18726b5f4 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:15.688791 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49774000-ba32-41f8-9cc3-bd088ed75bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:15.690481 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49774000-ba32-41f8-9cc3-bd088ed75bdb None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:15.691171 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49774000-ba32-41f8-9cc3-bd088ed75bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:15.692586 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49774000-ba32-41f8-9cc3-bd088ed75bdb None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:15.693713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49774000-ba32-41f8-9cc3-bd088ed75bdb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:15.694318 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49774000-ba32-41f8-9cc3-bd088ed75bdb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:15.695975 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49774000-ba32-41f8-9cc3-bd088ed75bdb None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:15.716399 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 386/766] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:31:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:15.718310 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8344a-9952-49bf-b4e2-6b96d7120880 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:15.719035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8344a-9952-49bf-b4e2-6b96d7120880 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:15.719534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec8344a-9952-49bf-b4e2-6b96d7120880 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:15.721943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 387/767] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:15.722871 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8344a-9952-49bf-b4e2-6b96d7120880 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:15.809316 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51910d74-1645-4f1d-b49c-6da18726b5f4 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:15.861254 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdb091-d783-43f4-bbb1-c4c62b677a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:15.862811 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdb091-d783-43f4-bbb1-c4c62b677a48 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:15.863109 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdb091-d783-43f4-bbb1-c4c62b677a48 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:15.863369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdb091-d783-43f4-bbb1-c4c62b677a48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:15.864214 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbdb091-d783-43f4-bbb1-c4c62b677a48 None admin] RBAC: Adding query filter params (role.id=11e7404599ae49afb5edd22dabe1a5b2, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=c700b0adb9f54947a88c1baa86accb9f) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:15.864529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbdb091-d783-43f4-bbb1-c4c62b677a48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:15.866273 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbdb091-d783-43f4-bbb1-c4c62b677a48 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:15.873076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 381/768] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:31:15 2022] GET /identity/v3/role_assignments?user.id=c700b0adb9f54947a88c1baa86accb9f&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=11e7404599ae49afb5edd22dabe1a5b2 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:15.874036 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdb091-d783-43f4-bbb1-c4c62b677a48 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:15.881578 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec8344a-9952-49bf-b4e2-6b96d7120880 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:15.928613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:15.936807 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01166802-2741-415d-a484-cef49f3e6fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:15.939399 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01166802-2741-415d-a484-cef49f3e6fbc None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:15.941292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01166802-2741-415d-a484-cef49f3e6fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:15.942558 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01166802-2741-415d-a484-cef49f3e6fbc None admin] PATH_INFO: `/v3/services/network` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:15.952522 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01166802-2741-415d-a484-cef49f3e6fbc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:15.952522 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01166802-2741-415d-a484-cef49f3e6fbc None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:15.965819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-01166802-2741-415d-a484-cef49f3e6fbc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 19:31:15.965819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:15.965819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:31:15.965819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:31:15.965819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:31:15.965819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:31:15.965819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:31:15.965819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:15.965819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 19:31:15.965819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:15.965819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:15.965819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:15.965819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:15.965819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:15.965819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:15.965819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:15.965819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:15.967160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:15.967160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:15.967160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:15.967160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:15.967160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:15.967160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:15.967160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:31:15.967160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:31:15.967160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:31:15.967160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:31:15.967160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:15.967160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:15.967160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:15.967160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:15.967160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:15.967160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:15.968020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:15.968020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:15.968020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:15.968020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:15.968020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:15.968020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:15.968020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:15.968020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:15.968020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:15.968020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:15.968020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:31:15.968020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:15.968020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 19:31:15.968020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:15.971127 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 388/769] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:15 2022] GET /identity/v3/services/network => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:15.972773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eacae3-26f5-48e3-ae19-dd6a849c029d None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:15.973297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eacae3-26f5-48e3-ae19-dd6a849c029d None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:15.973780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eacae3-26f5-48e3-ae19-dd6a849c029d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:15.995679 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-b1eacae3-26f5-48e3-ae19-dd6a849c029d None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:15.997301 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-b1eacae3-26f5-48e3-ae19-dd6a849c029d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:16.037181 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:16.054935 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdb091-d783-43f4-bbb1-c4c62b677a48 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:16.105710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228345e-82cc-4cf9-b755-f472cf4c7202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:16.106572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228345e-82cc-4cf9-b755-f472cf4c7202 None admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:16.106850 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228345e-82cc-4cf9-b755-f472cf4c7202 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:16.107124 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228345e-82cc-4cf9-b755-f472cf4c7202 None admin] PATH_INFO: `/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/7542e1fd160e4b2c9d25accf19aad19f/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:16.109341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7228345e-82cc-4cf9-b755-f472cf4c7202 None admin] RBAC: Authorizing `identity:create_grant(project_id=a47d6b6971c3452e9799e8146856bdc5, user_id=7542e1fd160e4b2c9d25accf19aad19f, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:16.110942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7228345e-82cc-4cf9-b755-f472cf4c7202 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:16.114794 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 389/770] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:16.117963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eacae3-26f5-48e3-ae19-dd6a849c029d None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:16.202539 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 382/771] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:31:15 2022] PUT /identity/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/7542e1fd160e4b2c9d25accf19aad19f/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:31:16.203707 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228345e-82cc-4cf9-b755-f472cf4c7202 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:16.306488 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eacae3-26f5-48e3-ae19-dd6a849c029d None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:16.396685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5cba1-028b-432b-a12d-f457791a3f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:16.396685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5cba1-028b-432b-a12d-f457791a3f5f None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:16.396685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5cba1-028b-432b-a12d-f457791a3f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:16.396685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5cba1-028b-432b-a12d-f457791a3f5f None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:16.397142 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5cba1-028b-432b-a12d-f457791a3f5f None admin] RBAC: Adding query filter params (name=network) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:16.397142 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5cba1-028b-432b-a12d-f457791a3f5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:16.397142 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5cba1-028b-432b-a12d-f457791a3f5f None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:16.407376 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 390/772] 10.222.0.13 () {56 vars in 1112 bytes} [Fri May 27 19:31:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:16.413071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5cba1-028b-432b-a12d-f457791a3f5f None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:16.433344 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:16.440120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228345e-82cc-4cf9-b755-f472cf4c7202 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:16.479985 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:16.497050 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1f048-2750-4774-bc46-2b18d71b1343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:16.498169 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1f048-2750-4774-bc46-2b18d71b1343 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:16.498169 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1f048-2750-4774-bc46-2b18d71b1343 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:16.498342 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc1f048-2750-4774-bc46-2b18d71b1343 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:16.498953 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1f048-2750-4774-bc46-2b18d71b1343 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:16.504257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc1f048-2750-4774-bc46-2b18d71b1343 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:16.523856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-bcc1f048-2750-4774-bc46-2b18d71b1343 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 19:31:16.523856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:16.523856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:31:16.523856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:31:16.523856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:31:16.523856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:31:16.523856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:31:16.523856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:16.523856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 19:31:16.523856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:16.523856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:16.523856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:16.523856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:16.523856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:16.523856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:16.523856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:16.524860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:16.524860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:16.524860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:16.524860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:16.524860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:16.524860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:16.524860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:16.524860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:31:16.524860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:31:16.524860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:31:16.524860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:31:16.524860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:16.524860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:16.524860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:16.524860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:16.524860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:16.525762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:16.525762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:16.525762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:16.525762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:16.525762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:16.525762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:16.525762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:16.525762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:16.525762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:16.525762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:16.525762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:16.525762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:31:16.525762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:16.525762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 19:31:16.525762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:16.525762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 383/773] 10.222.0.13 () {56 vars in 1129 bytes} [Fri May 27 19:31:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:16.534131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1f048-2750-4774-bc46-2b18d71b1343 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:16.619805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5cba1-028b-432b-a12d-f457791a3f5f None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:16.681049 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b904361-5838-4620-8c8a-e510523e2f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:16.681049 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b904361-5838-4620-8c8a-e510523e2f9c None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:16.681049 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b904361-5838-4620-8c8a-e510523e2f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:16.681049 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b904361-5838-4620-8c8a-e510523e2f9c None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:16.681462 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b904361-5838-4620-8c8a-e510523e2f9c None admin] RBAC: Adding query filter params (type=network) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:16.681543 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b904361-5838-4620-8c8a-e510523e2f9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:16.683070 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b904361-5838-4620-8c8a-e510523e2f9c None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:16.693565 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 391/774] 10.222.0.13 () {56 vars in 1112 bytes} [Fri May 27 19:31:16 2022] GET /identity/v3/services?type=network => generated 350 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:16.700416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b904361-5838-4620-8c8a-e510523e2f9c None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:16.740653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc1f048-2750-4774-bc46-2b18d71b1343 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:16.841771 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da9db5d-8a5b-4ef9-8e5b-e93bb4235a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:16.842922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9db5d-8a5b-4ef9-8e5b-e93bb4235a72 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:16.843435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9db5d-8a5b-4ef9-8e5b-e93bb4235a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:16.843916 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da9db5d-8a5b-4ef9-8e5b-e93bb4235a72 None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:16.844832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da9db5d-8a5b-4ef9-8e5b-e93bb4235a72 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:16.845385 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da9db5d-8a5b-4ef9-8e5b-e93bb4235a72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:16.847620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da9db5d-8a5b-4ef9-8e5b-e93bb4235a72 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:16.861245 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 384/775] 10.222.0.13 () {56 vars in 1124 bytes} [Fri May 27 19:31:16 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:16.874036 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da9db5d-8a5b-4ef9-8e5b-e93bb4235a72 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:16.906228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:16.922912 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b904361-5838-4620-8c8a-e510523e2f9c None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:16.964440 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:16.973994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41350f-7891-4851-ac09-66987453e930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:16.975721 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41350f-7891-4851-ac09-66987453e930 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:16.975721 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41350f-7891-4851-ac09-66987453e930 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:16.975721 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41350f-7891-4851-ac09-66987453e930 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:16.976037 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41350f-7891-4851-ac09-66987453e930 None admin] RBAC: Adding query filter params (interface=public, service_id=a715e217125048ca9b8492ffdbb309ab, region_id=RegionOne) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:16.976356 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41350f-7891-4851-ac09-66987453e930 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:16.977897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41350f-7891-4851-ac09-66987453e930 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:17.008276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 392/776] 10.222.0.13 () {56 vars in 1251 bytes} [Fri May 27 19:31:16 2022] GET /identity/v3/endpoints?service_id=a715e217125048ca9b8492ffdbb309ab&interface=public®ion_id=RegionOne => generated 194 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:17.018438 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41350f-7891-4851-ac09-66987453e930 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:17.146026 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da9db5d-8a5b-4ef9-8e5b-e93bb4235a72 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:17.193745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59d55c-ae49-48e9-bc87-1c6e192bfaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:17.194428 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59d55c-ae49-48e9-bc87-1c6e192bfaf0 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:17.194708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59d55c-ae49-48e9-bc87-1c6e192bfaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:17.200976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59d55c-ae49-48e9-bc87-1c6e192bfaf0 None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:17.200976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59d55c-ae49-48e9-bc87-1c6e192bfaf0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:17.200976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59d55c-ae49-48e9-bc87-1c6e192bfaf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:17.200976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59d55c-ae49-48e9-bc87-1c6e192bfaf0 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:17.209839 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 385/777] 10.222.0.13 () {56 vars in 1124 bytes} [Fri May 27 19:31:16 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:17.212157 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42365c07-4f32-4c2b-a07e-6c03af43b73a None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:17.212432 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42365c07-4f32-4c2b-a07e-6c03af43b73a None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:17.212714 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42365c07-4f32-4c2b-a07e-6c03af43b73a None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:17.222298 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 386/778] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:17.223239 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42365c07-4f32-4c2b-a07e-6c03af43b73a None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:17.253212 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41350f-7891-4851-ac09-66987453e930 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:17.329187 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1dddc4-bc69-478a-abfe-4b5b79d98a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:17.332484 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1dddc4-bc69-478a-abfe-4b5b79d98a16 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:17.332865 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1dddc4-bc69-478a-abfe-4b5b79d98a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:17.333170 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1dddc4-bc69-478a-abfe-4b5b79d98a16 None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:17.333919 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1dddc4-bc69-478a-abfe-4b5b79d98a16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:17.335533 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1dddc4-bc69-478a-abfe-4b5b79d98a16 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:17.344252 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 393/779] 10.222.0.13 () {56 vars in 1087 bytes} [Fri May 27 19:31:17 2022] GET /identity/v3/services => generated 772 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:17.345895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa46e3d-47dc-491d-b8e0-3c0d1832e5b7 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:17.352993 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa46e3d-47dc-491d-b8e0-3c0d1832e5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:17.352993 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa46e3d-47dc-491d-b8e0-3c0d1832e5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:17.357500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:17.372817 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42365c07-4f32-4c2b-a07e-6c03af43b73a None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:17.395409 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-3aa46e3d-47dc-491d-b8e0-3c0d1832e5b7 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:17.396670 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-3aa46e3d-47dc-491d-b8e0-3c0d1832e5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:17.422173 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517eb646-25e2-4a8c-b5a1-8cad6f69cdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:17.423290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517eb646-25e2-4a8c-b5a1-8cad6f69cdf6 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:17.423290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517eb646-25e2-4a8c-b5a1-8cad6f69cdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:17.424108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517eb646-25e2-4a8c-b5a1-8cad6f69cdf6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:17.425319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517eb646-25e2-4a8c-b5a1-8cad6f69cdf6 None admin] RBAC: Adding query filter params (interface=public, service_id=0200de4417854bb3b1ee155f8136329f, region_id=RegionOne) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:17.425646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517eb646-25e2-4a8c-b5a1-8cad6f69cdf6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:17.427243 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517eb646-25e2-4a8c-b5a1-8cad6f69cdf6 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:17.436579 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 387/780] 10.222.0.13 () {56 vars in 1251 bytes} [Fri May 27 19:31:17 2022] GET /identity/v3/endpoints?service_id=0200de4417854bb3b1ee155f8136329f&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:17.442622 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517eb646-25e2-4a8c-b5a1-8cad6f69cdf6 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:17.458512 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:17.594908 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517eb646-25e2-4a8c-b5a1-8cad6f69cdf6 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:17.643211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e53e1b-4a3a-4dc5-9e29-23e530176f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:17.645252 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e53e1b-4a3a-4dc5-9e29-23e530176f23 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:17.645883 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e53e1b-4a3a-4dc5-9e29-23e530176f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:17.646431 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e53e1b-4a3a-4dc5-9e29-23e530176f23 None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:17.647398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e53e1b-4a3a-4dc5-9e29-23e530176f23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:17.649131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e53e1b-4a3a-4dc5-9e29-23e530176f23 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:17.658624 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 388/781] 10.222.0.13 () {56 vars in 1087 bytes} [Fri May 27 19:31:17 2022] GET /identity/v3/services => generated 772 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:17.674344 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 394/782] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:17.699598 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e53e1b-4a3a-4dc5-9e29-23e530176f23 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:17.732843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:17.931578 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e53e1b-4a3a-4dc5-9e29-23e530176f23 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:17.982630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1bceb-6087-4ffe-91fe-259f3b0deeeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:17.983467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1bceb-6087-4ffe-91fe-259f3b0deeeb None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:17.983780 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1bceb-6087-4ffe-91fe-259f3b0deeeb None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:17.984065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1bceb-6087-4ffe-91fe-259f3b0deeeb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:18.003659 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1bceb-6087-4ffe-91fe-259f3b0deeeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:31:18.011943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-c9a1bceb-6087-4ffe-91fe-259f3b0deeeb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:31:18.011943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:18.011943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:18.011943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:18.011943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:18.011943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:18.011943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:18.011943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:18.011943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:18.011943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:18.011943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:18.011943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:18.011943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:31:18.011943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:31:18.011943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:31:18.011943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:31:18.012939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:31:18.012939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:31:18.012939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:31:18.012939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:31:18.012939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:18.012939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:18.012939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:18.012939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:18.012939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:18.012939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:18.012939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:18.012939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:18.012939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:18.012939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:18.012939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:18.012939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:18.014277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:18.014277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:18.014277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:18.014277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:18.014277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:31:18.014277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:31:18.014277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:31:18.014277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:31:18.014277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:31:18.014277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:31:18.014277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:31:18.014277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:18.014277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 389/783] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:31:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:18.020738 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa46e3d-47dc-491d-b8e0-3c0d1832e5b7 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:18.137759 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa46e3d-47dc-491d-b8e0-3c0d1832e5b7 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:18.209745 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fc0be-4508-46b9-b55e-811cb512e868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:18.210487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fc0be-4508-46b9-b55e-811cb512e868 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:18.210779 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fc0be-4508-46b9-b55e-811cb512e868 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:18.211058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fc0be-4508-46b9-b55e-811cb512e868 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:18.211846 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fc0be-4508-46b9-b55e-811cb512e868 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:18.213163 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fc0be-4508-46b9-b55e-811cb512e868 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:18.213582 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fc0be-4508-46b9-b55e-811cb512e868 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:18.228641 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 395/784] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:31:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:18.234408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1bceb-6087-4ffe-91fe-259f3b0deeeb None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:18.420582 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:18.486007 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1bceb-6087-4ffe-91fe-259f3b0deeeb None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:18.586610 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a2995-3a4a-42c6-82fb-afd4fa487cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:18.587428 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a2995-3a4a-42c6-82fb-afd4fa487cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:18.587709 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a2995-3a4a-42c6-82fb-afd4fa487cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:18.587985 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a2995-3a4a-42c6-82fb-afd4fa487cc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:18.621900 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02a2995-3a4a-42c6-82fb-afd4fa487cc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:18.623390 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02a2995-3a4a-42c6-82fb-afd4fa487cc6 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:18.653369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-c02a2995-3a4a-42c6-82fb-afd4fa487cc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:18.653369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:18.653369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:18.653369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:18.653369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:18.653369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:18.653369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:18.653369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:18.653369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:18.653369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:18.653369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:18.653369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:18.653369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:18.653369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:18.653369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:18.654345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:18.654345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:18.654345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:18.654345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:18.654345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:18.654345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:18.654345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:18.654345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:18.654345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:18.654345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:18.654345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:18.654345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:18.654345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:18.654345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:18.654345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:18.654345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:18.654345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:18.655284 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:18.655284 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:18.655284 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:18.655284 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:18.661352 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 390/785] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 427 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:18.663063 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fc0be-4508-46b9-b55e-811cb512e868 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:18.693685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:18.729242 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce51176-aa42-4ec2-8eb7-20578991fb4a None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:18.729574 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce51176-aa42-4ec2-8eb7-20578991fb4a None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:18.729807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce51176-aa42-4ec2-8eb7-20578991fb4a None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:18.731486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 391/786] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:18.738206 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79095293-ec79-45fb-96a2-504c78262c02 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:18.738419 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79095293-ec79-45fb-96a2-504c78262c02 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:18.738702 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79095293-ec79-45fb-96a2-504c78262c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:18.829187 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-79095293-ec79-45fb-96a2-504c78262c02 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:18.830592 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-79095293-ec79-45fb-96a2-504c78262c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:18.964110 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fc0be-4508-46b9-b55e-811cb512e868 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:18.994820 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 392/787] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:19.039970 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645898db-b4e4-41d8-a6ec-562e19d589bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:19.041363 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645898db-b4e4-41d8-a6ec-562e19d589bd None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:19.042041 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645898db-b4e4-41d8-a6ec-562e19d589bd None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:19.043251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79095293-ec79-45fb-96a2-504c78262c02 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:19.045794 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645898db-b4e4-41d8-a6ec-562e19d589bd None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:19.047636 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645898db-b4e4-41d8-a6ec-562e19d589bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:19.048184 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645898db-b4e4-41d8-a6ec-562e19d589bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:19.055506 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645898db-b4e4-41d8-a6ec-562e19d589bd None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:19.084378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 396/788] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:19.089978 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645898db-b4e4-41d8-a6ec-562e19d589bd None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:19.175669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:19.255420 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79095293-ec79-45fb-96a2-504c78262c02 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:19.293922 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:19.358333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ecafe6-96f4-4acd-abe1-ee640f28683a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:19.359611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecafe6-96f4-4acd-abe1-ee640f28683a None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:19.361892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecafe6-96f4-4acd-abe1-ee640f28683a None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:19.362618 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ecafe6-96f4-4acd-abe1-ee640f28683a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:19.369375 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645898db-b4e4-41d8-a6ec-562e19d589bd None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:19.392208 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ecafe6-96f4-4acd-abe1-ee640f28683a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:31:19.409576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-19ecafe6-96f4-4acd-abe1-ee640f28683a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 19:31:19.409576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:19.409576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:19.409576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:19.409576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:19.409576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:19.409576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:19.409576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:19.409576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:19.409576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:19.409576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:19.409576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:19.409576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:31:19.409576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:31:19.409576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:31:19.409576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:31:19.410562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:31:19.410562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:31:19.410562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:31:19.410562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:31:19.410562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:19.410562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:19.410562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:19.410562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:19.410562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:19.410562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:19.410562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:19.410562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:19.410562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:19.410562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:19.410562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:19.410562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:19.411792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:19.411792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:19.411792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:19.411792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:19.411792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:31:19.411792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:31:19.411792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:31:19.411792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:31:19.411792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:31:19.411792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:31:19.411792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 19:31:19.411792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:19.415855 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 393/789] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:31:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:19.419269 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34882778-b932-4553-951f-80f764438468 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:19.419558 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34882778-b932-4553-951f-80f764438468 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:19.419830 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34882778-b932-4553-951f-80f764438468 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:19.427642 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 394/790] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:19 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:19.428418 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34882778-b932-4553-951f-80f764438468 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:19.462909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a69449-1126-4734-b560-8f7175d8d2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:19.462909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a69449-1126-4734-b560-8f7175d8d2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:19.462909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a69449-1126-4734-b560-8f7175d8d2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:19.462909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a69449-1126-4734-b560-8f7175d8d2b6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:19.479066 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a69449-1126-4734-b560-8f7175d8d2b6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:19.480547 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a69449-1126-4734-b560-8f7175d8d2b6 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:19.656129 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34882778-b932-4553-951f-80f764438468 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:19.692365 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:19.728873 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4b6ec-fb94-4f64-9316-8a2f055090d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:19.729605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4b6ec-fb94-4f64-9316-8a2f055090d4 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:19.729938 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4b6ec-fb94-4f64-9316-8a2f055090d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:19.730157 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4b6ec-fb94-4f64-9316-8a2f055090d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:19.730969 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4b6ec-fb94-4f64-9316-8a2f055090d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:19.731274 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4b6ec-fb94-4f64-9316-8a2f055090d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:19.732666 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4b6ec-fb94-4f64-9316-8a2f055090d4 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:19.757431 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 395/791] 10.222.0.13 () {56 vars in 1103 bytes} [Fri May 27 19:31:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:19.759083 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893c373-52a9-4490-9c00-4b66c1e4c717 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:19.759371 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893c373-52a9-4490-9c00-4b66c1e4c717 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:19.759656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8893c373-52a9-4490-9c00-4b66c1e4c717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:19.789947 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-22a69449-1126-4734-b560-8f7175d8d2b6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 19:31:19.789947 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:19.789947 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:19.789947 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:19.789947 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:19.789947 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:19.789947 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:19.789947 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:19.789947 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:19.789947 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:19.789947 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:19.789947 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:19.789947 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:31:19.789947 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:31:19.789947 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:31:19.789947 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:31:19.790992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:19.790992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:19.790992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:31:19.790992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:19.790992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:31:19.790992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:19.790992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:19.790992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:19.790992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:19.790992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:19.790992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:19.790992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:19.790992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:19.790992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:19.790992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:19.790992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:19.790992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:19.790992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:19.791994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:19.791994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:19.791994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:31:19.791994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:31:19.791994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:31:19.791994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:31:19.791994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:31:19.791994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:31:19.791994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 19:31:19.791994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:19.799791 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 397/792] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:31:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 710 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:19.801654 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a69449-1126-4734-b560-8f7175d8d2b6 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:19.803245 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-8893c373-52a9-4490-9c00-4b66c1e4c717 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:19.804454 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-8893c373-52a9-4490-9c00-4b66c1e4c717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:19.968838 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a69449-1126-4734-b560-8f7175d8d2b6 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:19.982383 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 396/793] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:19.984564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893c373-52a9-4490-9c00-4b66c1e4c717 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:20.044731 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:20.053960 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392062f-6000-44a9-8b06-b35a9b31ce5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:20.055161 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392062f-6000-44a9-8b06-b35a9b31ce5c None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:20.055790 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392062f-6000-44a9-8b06-b35a9b31ce5c None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:20.056407 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b392062f-6000-44a9-8b06-b35a9b31ce5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:20.071311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392062f-6000-44a9-8b06-b35a9b31ce5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:20.073168 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b392062f-6000-44a9-8b06-b35a9b31ce5c None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:20.100623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-b392062f-6000-44a9-8b06-b35a9b31ce5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:20.100623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:20.100623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:20.100623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:20.100623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:20.100623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:20.100623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:20.100623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:20.100623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:20.100623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:20.100623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:20.100623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:20.100623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:20.100623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:20.100623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:20.101837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:20.101837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:20.101837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:20.101837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:20.101837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:20.101837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:20.101837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:20.101837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:20.101837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:20.101837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:20.101837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:20.101837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:20.101837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:20.101837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:20.101837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:20.101837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:20.101837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:20.102740 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:20.102740 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:20.102740 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:20.102740 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:20.105855 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 398/794] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:20.107191 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392062f-6000-44a9-8b06-b35a9b31ce5c None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:20.243389 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b392062f-6000-44a9-8b06-b35a9b31ce5c None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:20.249784 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8893c373-52a9-4490-9c00-4b66c1e4c717 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:20.271905 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:20.306835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b1784-b8dc-411d-b701-5e93632cc04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:20.307573 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b1784-b8dc-411d-b701-5e93632cc04b None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:20.307859 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b1784-b8dc-411d-b701-5e93632cc04b None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:20.308136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b1784-b8dc-411d-b701-5e93632cc04b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:20.319741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44020240-9c00-428c-997e-201549f53250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:20.320707 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44020240-9c00-428c-997e-201549f53250 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:20.321296 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44020240-9c00-428c-997e-201549f53250 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:20.321771 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44020240-9c00-428c-997e-201549f53250 None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:20.322798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44020240-9c00-428c-997e-201549f53250 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:20.323036 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229b1784-b8dc-411d-b701-5e93632cc04b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:31:20.323523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44020240-9c00-428c-997e-201549f53250 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:20.325186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44020240-9c00-428c-997e-201549f53250 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:20.333769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-229b1784-b8dc-411d-b701-5e93632cc04b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:31:20.333769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:20.333769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:20.333769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:20.333769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:20.333769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:20.333769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:20.333769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:20.333769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:20.333769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:20.333769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:20.333769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:20.333769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:31:20.333769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:31:20.333769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:31:20.333769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:31:20.334732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:31:20.334732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:31:20.334732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:31:20.334732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:31:20.334732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:20.334732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:20.334732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:20.334732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:20.334732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:20.334732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:20.334732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:20.334732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:20.334732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:20.334732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:20.334732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:20.334732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:20.335572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:20.335572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:20.335572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:20.335572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:20.335572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:31:20.335572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:31:20.335572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:31:20.335572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:31:20.335572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:31:20.335572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:31:20.335572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:31:20.335572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:20.340314 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 399/795] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:31:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:20.342402 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b1784-b8dc-411d-b701-5e93632cc04b None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:20.352346 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 397/796] 10.222.0.13 () {56 vars in 1135 bytes} [Fri May 27 19:31:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:20.354142 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71677c81-df0c-40c7-85b8-ba9c07a7bebb None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:20.354693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71677c81-df0c-40c7-85b8-ba9c07a7bebb None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:20.355192 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71677c81-df0c-40c7-85b8-ba9c07a7bebb None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:20.356939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 398/797] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:20.357917 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71677c81-df0c-40c7-85b8-ba9c07a7bebb None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:20.406757 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:20.489162 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229b1784-b8dc-411d-b701-5e93632cc04b None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:20.515269 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71677c81-df0c-40c7-85b8-ba9c07a7bebb None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:20.530517 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d0bcc-5f16-4277-8cdf-e5f8a138cb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:20.531554 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d0bcc-5f16-4277-8cdf-e5f8a138cb3e None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:20.531554 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d0bcc-5f16-4277-8cdf-e5f8a138cb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:20.531797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d0bcc-5f16-4277-8cdf-e5f8a138cb3e None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:20.533102 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d0bcc-5f16-4277-8cdf-e5f8a138cb3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:20.533469 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d0bcc-5f16-4277-8cdf-e5f8a138cb3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:20.536834 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d0bcc-5f16-4277-8cdf-e5f8a138cb3e None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:20.558294 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 400/798] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:20.559160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d0bcc-5f16-4277-8cdf-e5f8a138cb3e None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:20.582278 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee48aa9-c7fc-47f2-a564-7300e16c7889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:20.583340 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee48aa9-c7fc-47f2-a564-7300e16c7889 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:20.583631 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee48aa9-c7fc-47f2-a564-7300e16c7889 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:20.583914 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee48aa9-c7fc-47f2-a564-7300e16c7889 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:20.584678 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee48aa9-c7fc-47f2-a564-7300e16c7889 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:20.585031 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee48aa9-c7fc-47f2-a564-7300e16c7889 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:20.587710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee48aa9-c7fc-47f2-a564-7300e16c7889 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:20.597028 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 399/799] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:31:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:20.598388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e0da4-b4e0-4c2f-b150-7f4308ccb008 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:20.598681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e0da4-b4e0-4c2f-b150-7f4308ccb008 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:20.598962 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e0da4-b4e0-4c2f-b150-7f4308ccb008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:20.619859 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-e71e0da4-b4e0-4c2f-b150-7f4308ccb008 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:20.622002 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-e71e0da4-b4e0-4c2f-b150-7f4308ccb008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:20.653942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:20.709900 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d0bcc-5f16-4277-8cdf-e5f8a138cb3e None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:20.712066 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 400/800] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:20.713046 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e0da4-b4e0-4c2f-b150-7f4308ccb008 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:20.714370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:20.751744 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c5829-9bb1-4015-b183-a42530297043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:20.752474 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c5829-9bb1-4015-b183-a42530297043 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:20.752825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c5829-9bb1-4015-b183-a42530297043 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:20.753138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c5829-9bb1-4015-b183-a42530297043 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:20.763614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909c5829-9bb1-4015-b183-a42530297043 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:20.765166 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909c5829-9bb1-4015-b183-a42530297043 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:20.785331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-909c5829-9bb1-4015-b183-a42530297043 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:20.785331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:20.785331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:20.785331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:20.785331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:20.785331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:20.785331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:20.785331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:20.785331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:20.785331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:20.785331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:20.785331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:20.785331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:20.785331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:20.785331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:20.786529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:20.786529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:20.786529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:20.786529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:20.786529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:20.786529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:20.786529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:20.786529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:20.786529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:20.786529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:20.786529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:20.786529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:20.786529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:20.786529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:20.786529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:20.786529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:20.786529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:20.787597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:20.787597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:20.787597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:20.787597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:20.787597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 401/801] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:20.787597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c5829-9bb1-4015-b183-a42530297043 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:20.843771 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71e0da4-b4e0-4c2f-b150-7f4308ccb008 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:20.880669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5761c2-2445-4c03-be2d-937e127d2d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:20.881523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5761c2-2445-4c03-be2d-937e127d2d04 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:20.882169 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5761c2-2445-4c03-be2d-937e127d2d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:20.882169 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5761c2-2445-4c03-be2d-937e127d2d04 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:20.889571 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5761c2-2445-4c03-be2d-937e127d2d04 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:20.891033 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5761c2-2445-4c03-be2d-937e127d2d04 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:20.906467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-4e5761c2-2445-4c03-be2d-937e127d2d04 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 19:31:20.906467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:20.906467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:20.906467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:20.906467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:20.906467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:20.906467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:20.906467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:20.906467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:20.906467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:20.906467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:20.906467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:20.906467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:31:20.906467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:31:20.906467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:31:20.906467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:31:20.907681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:20.907681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:20.907681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:31:20.907681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:20.907681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:31:20.907681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:20.907681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:20.907681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:20.907681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:20.907681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:20.907681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:20.907681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:20.907681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:20.907681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:20.907681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:20.907681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:20.907681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:20.907681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:20.908643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:20.908643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:20.908643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:31:20.908643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:31:20.908643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:31:20.908643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:31:20.908643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:31:20.908643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:31:20.908643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 19:31:20.908643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:20.908643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 401/802] 10.222.0.13 () {56 vars in 1117 bytes} [Fri May 27 19:31:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:20.909692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb69419-7670-4c46-8599-4537f5c4cff3 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:20.909799 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c5829-9bb1-4015-b183-a42530297043 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:20.909887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb69419-7670-4c46-8599-4537f5c4cff3 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:20.909887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb69419-7670-4c46-8599-4537f5c4cff3 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:20.911100 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 402/803] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:20.911964 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb69419-7670-4c46-8599-4537f5c4cff3 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:20.931969 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:20.955622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f0252-4fcd-4394-9b51-be082bf93ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:20.956787 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f0252-4fcd-4394-9b51-be082bf93ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:20.957161 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f0252-4fcd-4394-9b51-be082bf93ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:20.957513 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0f0252-4fcd-4394-9b51-be082bf93ec4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:20.968371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f0252-4fcd-4394-9b51-be082bf93ec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:20.970198 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0f0252-4fcd-4394-9b51-be082bf93ec4 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:20.989626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-cd0f0252-4fcd-4394-9b51-be082bf93ec4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:20.989626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:20.989626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:20.989626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:20.989626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:20.989626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:20.989626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:20.989626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:20.989626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:20.989626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:20.989626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:20.989626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:20.989626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:20.989626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:20.989626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:20.990729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:20.990729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:20.990729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:20.990729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:20.990729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:20.990729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:20.990729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:20.990729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:20.990729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:20.990729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:20.990729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:20.990729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:20.990729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:20.990729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:20.990729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:20.990729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:20.990729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:20.991779 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:20.991779 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:20.991779 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:20.991779 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:20.991779 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 402/804] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:20.992325 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f0252-4fcd-4394-9b51-be082bf93ec4 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:21.005348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:21.025384 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb69419-7670-4c46-8599-4537f5c4cff3 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:21.061780 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5935d99-e882-4c8c-9579-b1daa033bf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:21.062613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5935d99-e882-4c8c-9579-b1daa033bf63 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:21.062900 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5935d99-e882-4c8c-9579-b1daa033bf63 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:21.063176 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5935d99-e882-4c8c-9579-b1daa033bf63 None admin] PATH_INFO: `/v3/services/network` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:21.063938 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5935d99-e882-4c8c-9579-b1daa033bf63 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:21.065929 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5935d99-e882-4c8c-9579-b1daa033bf63 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:21.085653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-c5935d99-e882-4c8c-9579-b1daa033bf63 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 19:31:21.085653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:21.085653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:31:21.085653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:31:21.085653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:31:21.085653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:31:21.085653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:31:21.085653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:21.085653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 19:31:21.085653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:21.085653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:21.085653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:21.085653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:21.085653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:21.085653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:21.085653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:21.085653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:21.086892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:21.086892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:21.086892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:21.086892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:21.086892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:21.086892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:21.086892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:31:21.086892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:31:21.086892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:31:21.086892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:31:21.086892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:21.086892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:21.086892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:21.086892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:21.086892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:21.086892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:21.087920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:21.087920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:21.087920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:21.087920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:21.087920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:21.087920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:21.087920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:21.087920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:21.087920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:21.087920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:21.087920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:31:21.087920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:21.087920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 19:31:21.087920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:21.087920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 403/805] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:20 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:21.088909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5935d99-e882-4c8c-9579-b1daa033bf63 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:21.114013 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0f0252-4fcd-4394-9b51-be082bf93ec4 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:21.154195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b68872-6583-4d84-a314-ad35baed02d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:21.154955 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b68872-6583-4d84-a314-ad35baed02d7 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:21.155303 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b68872-6583-4d84-a314-ad35baed02d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:21.155650 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b68872-6583-4d84-a314-ad35baed02d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:21.156473 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b68872-6583-4d84-a314-ad35baed02d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:21.156821 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b68872-6583-4d84-a314-ad35baed02d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:21.159097 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b68872-6583-4d84-a314-ad35baed02d7 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:21.170502 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 403/806] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:21.171750 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbe6ed-1b7f-459e-9c55-09c3b42dc4b5 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:21.172040 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbe6ed-1b7f-459e-9c55-09c3b42dc4b5 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:21.172414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fbe6ed-1b7f-459e-9c55-09c3b42dc4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:21.176324 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:21.190240 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-f0fbe6ed-1b7f-459e-9c55-09c3b42dc4b5 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:21.191418 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-f0fbe6ed-1b7f-459e-9c55-09c3b42dc4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:21.199000 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5935d99-e882-4c8c-9579-b1daa033bf63 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:21.239427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2adf62-f32a-4da7-bf72-ac79d04f8803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:21.240103 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2adf62-f32a-4da7-bf72-ac79d04f8803 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:21.240385 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2adf62-f32a-4da7-bf72-ac79d04f8803 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:21.240664 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2adf62-f32a-4da7-bf72-ac79d04f8803 None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:21.241634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2adf62-f32a-4da7-bf72-ac79d04f8803 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:21.241994 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2adf62-f32a-4da7-bf72-ac79d04f8803 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:21.243433 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2adf62-f32a-4da7-bf72-ac79d04f8803 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:21.264091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 404/807] 10.222.0.13 () {56 vars in 1149 bytes} [Fri May 27 19:31:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:21.265156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2adf62-f32a-4da7-bf72-ac79d04f8803 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:21.268553 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:21.274830 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 404/808] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:21.288823 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fbe6ed-1b7f-459e-9c55-09c3b42dc4b5 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:21.382311 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2adf62-f32a-4da7-bf72-ac79d04f8803 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:21.405452 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fbe6ed-1b7f-459e-9c55-09c3b42dc4b5 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:21.416979 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ced894-963f-45ce-8300-92faced5c82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:21.417635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ced894-963f-45ce-8300-92faced5c82d None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:21.417920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ced894-963f-45ce-8300-92faced5c82d None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:21.418199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ced894-963f-45ce-8300-92faced5c82d None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:21.418919 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ced894-963f-45ce-8300-92faced5c82d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:21.419227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ced894-963f-45ce-8300-92faced5c82d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:21.420611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ced894-963f-45ce-8300-92faced5c82d None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:21.429209 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:21.432719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 405/809] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:21.433368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ced894-963f-45ce-8300-92faced5c82d None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:21.444457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38552ff0-fdaa-45b9-b065-d6c304503c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:21.445351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38552ff0-fdaa-45b9-b065-d6c304503c85 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:21.445698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38552ff0-fdaa-45b9-b065-d6c304503c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:21.446039 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38552ff0-fdaa-45b9-b065-d6c304503c85 None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:21.446816 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38552ff0-fdaa-45b9-b065-d6c304503c85 None admin] RBAC: Adding query filter params (name=network) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:21.447187 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38552ff0-fdaa-45b9-b065-d6c304503c85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:21.448642 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38552ff0-fdaa-45b9-b065-d6c304503c85 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:21.453227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 405/810] 10.222.0.13 () {56 vars in 1112 bytes} [Fri May 27 19:31:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:21.454017 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38552ff0-fdaa-45b9-b065-d6c304503c85 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:21.515899 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:21.538785 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ced894-963f-45ce-8300-92faced5c82d None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:21.554129 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38552ff0-fdaa-45b9-b065-d6c304503c85 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:21.573249 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96765111-8924-4850-a230-e7d3f5fb83a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:21.574044 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96765111-8924-4850-a230-e7d3f5fb83a2 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:21.574324 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96765111-8924-4850-a230-e7d3f5fb83a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:21.574596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96765111-8924-4850-a230-e7d3f5fb83a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:21.582068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96765111-8924-4850-a230-e7d3f5fb83a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:21.583504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96765111-8924-4850-a230-e7d3f5fb83a2 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:21.589800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b898a-68a8-4b8e-93ec-8054ae9188b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:21.590593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b898a-68a8-4b8e-93ec-8054ae9188b2 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:21.590878 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b898a-68a8-4b8e-93ec-8054ae9188b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:21.591156 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b898a-68a8-4b8e-93ec-8054ae9188b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:21.597500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-96765111-8924-4850-a230-e7d3f5fb83a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:21.597500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:21.597500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:21.597500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:21.597500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:21.597500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:21.597500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:21.597500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:21.597500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:21.597500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:21.597500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:21.597500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:21.597500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:31:21.597500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:31:21.597500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:31:21.598529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:31:21.598529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:21.598529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:21.598529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:21.598529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:21.598529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:21.598529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:21.598529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:21.598529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:21.598529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:21.598529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:21.598529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:21.598529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:21.598529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:21.598529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:21.598529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:21.598529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:21.599790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:31:21.599790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:31:21.599790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:31:21.599790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:31:21.599790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:31:21.599790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:31:21.599790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:21.599790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:21.599790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 406/811] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:21.599790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96765111-8924-4850-a230-e7d3f5fb83a2 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:21.604715 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b898a-68a8-4b8e-93ec-8054ae9188b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:21.606205 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047b898a-68a8-4b8e-93ec-8054ae9188b2 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:21.624509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-047b898a-68a8-4b8e-93ec-8054ae9188b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:21.624509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:21.624509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:21.624509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:21.624509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:21.624509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:21.624509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:21.624509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:21.624509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:21.624509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:21.624509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:21.624509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:21.624509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:21.624509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:21.624509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:21.625662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:21.625662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:21.625662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:21.625662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:21.625662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:21.625662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:21.625662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:21.625662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:21.625662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:21.625662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:21.625662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:21.625662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:21.625662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:21.625662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:21.625662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:21.625662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:21.625662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:21.627275 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:21.627275 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:21.627275 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:21.627275 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:21.627275 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 406/812] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:21.627557 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b898a-68a8-4b8e-93ec-8054ae9188b2 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:21.664685 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:21.707137 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96765111-8924-4850-a230-e7d3f5fb83a2 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:21.726714 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047b898a-68a8-4b8e-93ec-8054ae9188b2 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:21.743141 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:21.763306 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9ec90-10d4-4006-bca3-68191218925b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:21.763480 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa6f15-ce90-417c-931f-a03ea5c1b775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:21.764107 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9ec90-10d4-4006-bca3-68191218925b None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:21.764350 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa6f15-ce90-417c-931f-a03ea5c1b775 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:21.764441 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9ec90-10d4-4006-bca3-68191218925b None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:21.764727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa6f15-ce90-417c-931f-a03ea5c1b775 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:21.764838 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9ec90-10d4-4006-bca3-68191218925b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:21.764977 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa6f15-ce90-417c-931f-a03ea5c1b775 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:21.765632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa6f15-ce90-417c-931f-a03ea5c1b775 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:21.767489 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aa6f15-ce90-417c-931f-a03ea5c1b775 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:21.772131 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9ec90-10d4-4006-bca3-68191218925b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:21.773707 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9ec90-10d4-4006-bca3-68191218925b None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:21.779741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-f0aa6f15-ce90-417c-931f-a03ea5c1b775 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 19:31:21.779741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:21.779741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:31:21.779741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:31:21.779741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:31:21.779741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:31:21.779741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:31:21.779741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:21.779741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 19:31:21.779741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:21.779741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:21.779741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:21.779741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:21.779741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:21.779741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:21.779741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:21.780736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:21.780736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:21.780736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:21.780736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:21.780736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:21.780736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:21.780736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:21.780736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:31:21.780736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:31:21.780736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:31:21.780736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:31:21.780736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:21.780736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:21.780736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:21.780736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:21.780736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:21.781808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:21.781808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:21.781808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:21.781808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:21.781808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:21.781808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:21.781808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:21.781808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:21.781808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:21.781808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:21.781808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:21.781808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:31:21.781808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:21.781808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 19:31:21.781808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:21.781808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 407/813] 10.222.0.13 () {56 vars in 1129 bytes} [Fri May 27 19:31:21 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:21.783188 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa6f15-ce90-417c-931f-a03ea5c1b775 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:21.791944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-aca9ec90-10d4-4006-bca3-68191218925b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 19:31:21.791944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:21.791944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:21.791944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:21.791944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:21.791944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:21.791944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:21.791944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:21.791944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:21.791944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:21.791944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:21.791944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:21.791944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:31:21.791944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:31:21.791944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:31:21.791944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:31:21.793092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:21.793092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:21.793092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:31:21.793092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:21.793092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:31:21.793092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:21.793092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:21.793092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:21.793092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:21.793092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:21.793092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:21.793092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:21.793092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:21.793092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:21.793092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:21.793092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:21.793092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:21.793092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:21.794101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:21.794101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:21.794101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:31:21.794101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:31:21.794101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:31:21.794101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:31:21.794101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:31:21.794101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:31:21.794101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 19:31:21.794101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:21.794101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 407/814] 10.222.0.13 () {56 vars in 1093 bytes} [Fri May 27 19:31:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:21.794101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9ec90-10d4-4006-bca3-68191218925b None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:21.888240 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aa6f15-ce90-417c-931f-a03ea5c1b775 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:21.895740 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9ec90-10d4-4006-bca3-68191218925b None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:21.913162 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:21.930695 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7606515-9bdc-47ff-a172-308042414294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:21.930695 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7606515-9bdc-47ff-a172-308042414294 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:21.930695 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7606515-9bdc-47ff-a172-308042414294 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:21.930695 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7606515-9bdc-47ff-a172-308042414294 None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:21.931319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7606515-9bdc-47ff-a172-308042414294 None admin] RBAC: Adding query filter params (type=network) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:21.931319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7606515-9bdc-47ff-a172-308042414294 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:21.931319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7606515-9bdc-47ff-a172-308042414294 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:21.931576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa4446-74a6-4d23-8eee-8af33b847948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:21.931576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa4446-74a6-4d23-8eee-8af33b847948 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:21.931576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa4446-74a6-4d23-8eee-8af33b847948 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:21.931576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa4446-74a6-4d23-8eee-8af33b847948 None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:21.931919 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa4446-74a6-4d23-8eee-8af33b847948 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:21.931919 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa4446-74a6-4d23-8eee-8af33b847948 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:21.932665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 408/815] 10.222.0.13 () {56 vars in 1112 bytes} [Fri May 27 19:31:21 2022] GET /identity/v3/services?type=network => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:21.932836 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa4446-74a6-4d23-8eee-8af33b847948 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:21.933501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7606515-9bdc-47ff-a172-308042414294 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:21.948665 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 408/816] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:31:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:21.949538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa4446-74a6-4d23-8eee-8af33b847948 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:21.983805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:22.042947 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7606515-9bdc-47ff-a172-308042414294 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:22.049454 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa4446-74a6-4d23-8eee-8af33b847948 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:22.077494 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a2b82-0428-4bdf-bddd-e23fe18d3459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:22.078337 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a2b82-0428-4bdf-bddd-e23fe18d3459 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:22.078621 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a2b82-0428-4bdf-bddd-e23fe18d3459 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:22.079028 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a2b82-0428-4bdf-bddd-e23fe18d3459 None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:22.079940 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a2b82-0428-4bdf-bddd-e23fe18d3459 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:22.080252 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a2b82-0428-4bdf-bddd-e23fe18d3459 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:22.081930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a2b82-0428-4bdf-bddd-e23fe18d3459 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:22.084002 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e1dc3-b3e1-440a-a878-6dfaff5d9fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:22.084661 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e1dc3-b3e1-440a-a878-6dfaff5d9fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:22.084938 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e1dc3-b3e1-440a-a878-6dfaff5d9fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:22.085238 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e1dc3-b3e1-440a-a878-6dfaff5d9fd3 None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:22.085937 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e1dc3-b3e1-440a-a878-6dfaff5d9fd3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:22.086240 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e1dc3-b3e1-440a-a878-6dfaff5d9fd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:22.087583 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e1dc3-b3e1-440a-a878-6dfaff5d9fd3 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:22.091925 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 409/817] 10.222.0.13 () {56 vars in 1124 bytes} [Fri May 27 19:31:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:22.092695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e1dc3-b3e1-440a-a878-6dfaff5d9fd3 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:22.099441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 409/818] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:22.099441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a2b82-0428-4bdf-bddd-e23fe18d3459 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:22.144838 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:22.191560 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:22.195447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e1dc3-b3e1-440a-a878-6dfaff5d9fd3 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:22.196035 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a2b82-0428-4bdf-bddd-e23fe18d3459 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:22.229401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ac956-bf48-4add-b95f-b1f9741190a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:22.230589 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ac956-bf48-4add-b95f-b1f9741190a4 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:22.231103 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ac956-bf48-4add-b95f-b1f9741190a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:22.231918 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8ac956-bf48-4add-b95f-b1f9741190a4 None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:22.232934 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de4c25-fcb9-43b6-ae53-9379a8c9af56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:22.233128 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ac956-bf48-4add-b95f-b1f9741190a4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:22.233446 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ac956-bf48-4add-b95f-b1f9741190a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:22.233627 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de4c25-fcb9-43b6-ae53-9379a8c9af56 None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:22.233912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de4c25-fcb9-43b6-ae53-9379a8c9af56 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:22.234192 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de4c25-fcb9-43b6-ae53-9379a8c9af56 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:22.234882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8ac956-bf48-4add-b95f-b1f9741190a4 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:22.234997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de4c25-fcb9-43b6-ae53-9379a8c9af56 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:22.236306 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de4c25-fcb9-43b6-ae53-9379a8c9af56 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:22.253068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 410/819] 10.222.0.13 () {56 vars in 1133 bytes} [Fri May 27 19:31:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:22.253882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ac956-bf48-4add-b95f-b1f9741190a4 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:22.343055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 410/820] 10.222.0.13 () {60 vars in 1144 bytes} [Fri May 27 19:31:22 2022] POST /identity/v3/endpoints => generated 327 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:31:22.343896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de4c25-fcb9-43b6-ae53-9379a8c9af56 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:22.355997 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8ac956-bf48-4add-b95f-b1f9741190a4 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:22.408813 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1732f-e3f2-4af0-b709-cb3b42c99d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:22.409767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1732f-e3f2-4af0-b709-cb3b42c99d08 None admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:22.410051 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1732f-e3f2-4af0-b709-cb3b42c99d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:22.410328 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1732f-e3f2-4af0-b709-cb3b42c99d08 None admin] PATH_INFO: `/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/37bc6d0abb454de19e1ab2f46bf4a593/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:22.413004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1732f-e3f2-4af0-b709-cb3b42c99d08 None admin] RBAC: Authorizing `identity:create_grant(project_id=a47d6b6971c3452e9799e8146856bdc5, user_id=37bc6d0abb454de19e1ab2f46bf4a593, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:22.414496 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1732f-e3f2-4af0-b709-cb3b42c99d08 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:22.472282 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de4c25-fcb9-43b6-ae53-9379a8c9af56 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:22.476327 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:22.494474 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 411/821] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:31:22 2022] PUT /identity/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/37bc6d0abb454de19e1ab2f46bf4a593/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:31:22.495707 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1732f-e3f2-4af0-b709-cb3b42c99d08 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:22.517191 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:22.569506 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cf23e-155d-4c4b-b704-603adb02e125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:22.570656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cf23e-155d-4c4b-b704-603adb02e125 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:22.571339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cf23e-155d-4c4b-b704-603adb02e125 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:22.571809 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cf23e-155d-4c4b-b704-603adb02e125 None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:22.573066 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cf23e-155d-4c4b-b704-603adb02e125 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:22.573582 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cf23e-155d-4c4b-b704-603adb02e125 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:22.577683 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cf23e-155d-4c4b-b704-603adb02e125 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:22.584657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 411/822] 10.222.0.13 () {56 vars in 1124 bytes} [Fri May 27 19:31:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:22.587971 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cf23e-155d-4c4b-b704-603adb02e125 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:22.665938 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1732f-e3f2-4af0-b709-cb3b42c99d08 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:22.710649 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5e9be-21ed-41f8-bff0-4554e66c8ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:22.711598 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5e9be-21ed-41f8-bff0-4554e66c8ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:22.711941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5e9be-21ed-41f8-bff0-4554e66c8ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:22.712357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5e9be-21ed-41f8-bff0-4554e66c8ae9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:22.723759 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5e9be-21ed-41f8-bff0-4554e66c8ae9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:22.724805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5e9be-21ed-41f8-bff0-4554e66c8ae9 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:22.740157 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cf23e-155d-4c4b-b704-603adb02e125 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:22.745845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-6fc5e9be-21ed-41f8-bff0-4554e66c8ae9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:22.745845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:22.745845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:22.745845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:22.745845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:22.745845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:22.745845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:22.745845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:22.745845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:22.745845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:22.745845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:22.745845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:22.745845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:31:22.745845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:31:22.745845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:31:22.747467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:31:22.747467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:22.747467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:22.747467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:22.747467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:22.747467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:22.747467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:22.747467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:22.747467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:22.747467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:22.747467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:22.747467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:22.747467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:22.747467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:22.747467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:22.747467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:22.747467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:22.749034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:31:22.749034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:31:22.749034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:31:22.749034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:31:22.749034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:31:22.749034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:31:22.749034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:22.749034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:22.749034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 412/823] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:22.749546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5e9be-21ed-41f8-bff0-4554e66c8ae9 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:22.793175 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b3a48-61b3-4726-9997-2df736f111c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:22.794434 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b3a48-61b3-4726-9997-2df736f111c7 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:22.795057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b3a48-61b3-4726-9997-2df736f111c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:22.795502 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8b3a48-61b3-4726-9997-2df736f111c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:22.836285 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:22.841671 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:22.872280 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5e9be-21ed-41f8-bff0-4554e66c8ae9 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:22.880308 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b3a48-61b3-4726-9997-2df736f111c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:22.882075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8b3a48-61b3-4726-9997-2df736f111c7 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:22.910939 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fc29c-b18a-4b31-9a32-702e1913b1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:22.911619 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fc29c-b18a-4b31-9a32-702e1913b1c5 None admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:22.911903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fc29c-b18a-4b31-9a32-702e1913b1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:22.912183 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fc29c-b18a-4b31-9a32-702e1913b1c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:22.912930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9fc29c-b18a-4b31-9a32-702e1913b1c5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:22.914343 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9fc29c-b18a-4b31-9a32-702e1913b1c5 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:22.994186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 412/824] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:31:22 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 19:31:22.994981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b3a48-61b3-4726-9997-2df736f111c7 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:23.016981 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 413/825] 10.222.0.13 () {60 vars in 1144 bytes} [Fri May 27 19:31:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:31:23.016981 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fc29c-b18a-4b31-9a32-702e1913b1c5 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:23.114957 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:23.184192 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8b3a48-61b3-4726-9997-2df736f111c7 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:23.203085 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fc29c-b18a-4b31-9a32-702e1913b1c5 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:23.233548 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce7519-89fb-4efb-b9c9-77574f34a284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:23.234234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce7519-89fb-4efb-b9c9-77574f34a284 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:23.234515 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce7519-89fb-4efb-b9c9-77574f34a284 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:23.234789 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce7519-89fb-4efb-b9c9-77574f34a284 None admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:23.236489 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ce7519-89fb-4efb-b9c9-77574f34a284 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:23.236882 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ce7519-89fb-4efb-b9c9-77574f34a284 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:23.238388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ce7519-89fb-4efb-b9c9-77574f34a284 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:23.238734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:23.253106 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df876d4-21f3-4c5c-8bd9-f2b46119e34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:23.253770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 413/826] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:31:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:23.253894 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df876d4-21f3-4c5c-8bd9-f2b46119e34d None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:23.254184 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df876d4-21f3-4c5c-8bd9-f2b46119e34d None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:23.254467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df876d4-21f3-4c5c-8bd9-f2b46119e34d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:23.259748 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce7519-89fb-4efb-b9c9-77574f34a284 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:23.268299 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df876d4-21f3-4c5c-8bd9-f2b46119e34d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:23.269906 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df876d4-21f3-4c5c-8bd9-f2b46119e34d None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:23.290767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-6df876d4-21f3-4c5c-8bd9-f2b46119e34d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:23.290767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:23.290767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:23.290767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:23.290767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:23.290767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:23.290767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:23.290767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:23.290767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:23.290767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:23.290767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:23.290767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:23.290767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:23.290767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:23.290767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:23.291869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:23.291869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:23.291869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:23.291869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:23.291869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:23.291869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:23.291869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:23.291869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:23.291869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:23.291869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:23.291869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:23.291869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:23.291869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:23.291869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:23.291869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:23.291869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:23.291869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:23.298040 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:23.298040 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:23.298040 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:23.298040 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:23.298040 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 414/827] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:23.299778 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df876d4-21f3-4c5c-8bd9-f2b46119e34d None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:23.419239 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df876d4-21f3-4c5c-8bd9-f2b46119e34d None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:23.425264 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce7519-89fb-4efb-b9c9-77574f34a284 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:23.457519 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10200f64-8ab6-4b77-b9ae-ff1eae2568cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:23.458002 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10200f64-8ab6-4b77-b9ae-ff1eae2568cf None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:23.458273 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10200f64-8ab6-4b77-b9ae-ff1eae2568cf None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:23.458583 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10200f64-8ab6-4b77-b9ae-ff1eae2568cf None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:23.459284 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10200f64-8ab6-4b77-b9ae-ff1eae2568cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:23.459585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10200f64-8ab6-4b77-b9ae-ff1eae2568cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:23.460980 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10200f64-8ab6-4b77-b9ae-ff1eae2568cf None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:23.464018 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:23.474154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e863f1-b1c5-485a-8c92-39a72ea637c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:23.475415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e863f1-b1c5-485a-8c92-39a72ea637c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:23.475897 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e863f1-b1c5-485a-8c92-39a72ea637c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:23.476384 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e863f1-b1c5-485a-8c92-39a72ea637c0 None admin] PATH_INFO: `/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/c700b0adb9f54947a88c1baa86accb9f/roles/11e7404599ae49afb5edd22dabe1a5b2` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:23.477925 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 415/828] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:23.483070 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10200f64-8ab6-4b77-b9ae-ff1eae2568cf None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:23.497827 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e863f1-b1c5-485a-8c92-39a72ea637c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=a47d6b6971c3452e9799e8146856bdc5, user_id=c700b0adb9f54947a88c1baa86accb9f, role_id=11e7404599ae49afb5edd22dabe1a5b2)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:23.499639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e863f1-b1c5-485a-8c92-39a72ea637c0 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:23.525496 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:23.589273 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 414/829] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:31:23 2022] PUT /identity/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/c700b0adb9f54947a88c1baa86accb9f/roles/11e7404599ae49afb5edd22dabe1a5b2 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:31:23.603079 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10200f64-8ab6-4b77-b9ae-ff1eae2568cf None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:23.649936 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0229f-66bf-4f22-b12e-c84cbc6fa4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:23.650756 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0229f-66bf-4f22-b12e-c84cbc6fa4ff None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:23.651037 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0229f-66bf-4f22-b12e-c84cbc6fa4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:23.651315 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0229f-66bf-4f22-b12e-c84cbc6fa4ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:23.660293 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d0229f-66bf-4f22-b12e-c84cbc6fa4ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:23.661972 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d0229f-66bf-4f22-b12e-c84cbc6fa4ff None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:23.679061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-39d0229f-66bf-4f22-b12e-c84cbc6fa4ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:23.679061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:23.679061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:23.679061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:23.679061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:23.679061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:23.679061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:23.679061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:23.679061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:23.679061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:23.679061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:23.679061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:23.679061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:31:23.679061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:31:23.679061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:31:23.680174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:31:23.680174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:23.680174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:23.680174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:23.680174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:23.680174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:23.680174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:23.680174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:23.680174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:23.680174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:23.680174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:23.680174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:23.680174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:23.680174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:23.680174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:23.680174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:23.680174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:23.681125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:31:23.681125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:31:23.681125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:31:23.681125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:31:23.681125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:31:23.681125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:31:23.681125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:23.681125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:23.681125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 416/830] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:31:23.686719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e863f1-b1c5-485a-8c92-39a72ea637c0 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:23.815484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e863f1-b1c5-485a-8c92-39a72ea637c0 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:23.878217 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26fbdc5-5ba1-43e7-a060-42677b357005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:23.879273 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fbdc5-5ba1-43e7-a060-42677b357005 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:23.879712 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fbdc5-5ba1-43e7-a060-42677b357005 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:23.879991 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26fbdc5-5ba1-43e7-a060-42677b357005 None admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:23.880786 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26fbdc5-5ba1-43e7-a060-42677b357005 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:23.881758 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26fbdc5-5ba1-43e7-a060-42677b357005 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:23.882683 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26fbdc5-5ba1-43e7-a060-42677b357005 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:23.900764 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 415/831] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:31:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:23.902191 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0229f-66bf-4f22-b12e-c84cbc6fa4ff None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:24.002936 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:24.009748 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0229f-66bf-4f22-b12e-c84cbc6fa4ff None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:24.044846 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e2eb5-356d-4dcc-b0df-62d893b03e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:24.045964 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e2eb5-356d-4dcc-b0df-62d893b03e60 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:24.046447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e2eb5-356d-4dcc-b0df-62d893b03e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:24.046917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e2eb5-356d-4dcc-b0df-62d893b03e60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:24.048573 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e2eb5-356d-4dcc-b0df-62d893b03e60 None admin] RBAC: Adding query filter params (role.id=6505bbb129404436966ee59aa7a01d8e, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=7542e1fd160e4b2c9d25accf19aad19f) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:24.049140 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e2eb5-356d-4dcc-b0df-62d893b03e60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:24.050945 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e2eb5-356d-4dcc-b0df-62d893b03e60 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:24.057832 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 417/832] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:31:23 2022] GET /identity/v3/role_assignments?user.id=7542e1fd160e4b2c9d25accf19aad19f&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=6505bbb129404436966ee59aa7a01d8e => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:24.915955 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88556f64-a9f0-4cca-95e2-f13470a36505 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:24.916457 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88556f64-a9f0-4cca-95e2-f13470a36505 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:24.916457 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88556f64-a9f0-4cca-95e2-f13470a36505 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:24.918128 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 416/833] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:24.925016 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f4389-6de5-426a-ba57-c1c8f297dc04 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:24.925542 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f4389-6de5-426a-ba57-c1c8f297dc04 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:24.925768 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545f4389-6de5-426a-ba57-c1c8f297dc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:24.943745 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-545f4389-6de5-426a-ba57-c1c8f297dc04 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:24.945033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-545f4389-6de5-426a-ba57-c1c8f297dc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:25.050026 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 418/834] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:24 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:25.074095 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88556f64-a9f0-4cca-95e2-f13470a36505 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:25.118931 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:25.189630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88556f64-a9f0-4cca-95e2-f13470a36505 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:25.229175 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed4ddc-fbf0-4655-a9cf-1493b475e400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:25.229989 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed4ddc-fbf0-4655-a9cf-1493b475e400 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:25.230276 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed4ddc-fbf0-4655-a9cf-1493b475e400 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:25.230558 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed4ddc-fbf0-4655-a9cf-1493b475e400 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:25.247118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed4ddc-fbf0-4655-a9cf-1493b475e400 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:31:25.254895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-c2ed4ddc-fbf0-4655-a9cf-1493b475e400 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:31:25.254895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:25.254895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:25.254895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:25.254895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:25.254895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:25.254895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:25.254895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:25.254895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:25.254895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:25.254895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:25.254895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:25.254895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:31:25.254895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:31:25.254895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:31:25.254895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:31:25.255854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:31:25.255854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:31:25.255854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:31:25.255854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:31:25.255854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:25.255854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:25.255854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:25.255854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:25.255854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:25.255854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:25.255854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:25.255854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:25.255854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:25.255854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:25.255854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:25.255854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:25.256728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:25.256728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:25.256728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:25.256728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:25.256728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:31:25.256728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:31:25.256728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:31:25.256728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:31:25.256728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:31:25.256728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:31:25.256728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:31:25.256728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:25.256728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 417/835] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:31:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:25.262216 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f4389-6de5-426a-ba57-c1c8f297dc04 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:25.381529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:25.392065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545f4389-6de5-426a-ba57-c1c8f297dc04 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:25.428503 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba042d-ae26-4cf3-b1f1-942ce8acf617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:25.429345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba042d-ae26-4cf3-b1f1-942ce8acf617 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:25.429625 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba042d-ae26-4cf3-b1f1-942ce8acf617 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:25.429898 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba042d-ae26-4cf3-b1f1-942ce8acf617 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:25.430716 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba042d-ae26-4cf3-b1f1-942ce8acf617 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:25.431025 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba042d-ae26-4cf3-b1f1-942ce8acf617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:25.432469 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba042d-ae26-4cf3-b1f1-942ce8acf617 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:25.446689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 419/836] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:31:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:25.448101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed4ddc-fbf0-4655-a9cf-1493b475e400 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:25.554209 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b2e59-a6c2-45d9-ad46-eb5442975e90 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:25.555227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b2e59-a6c2-45d9-ad46-eb5442975e90 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:25.555227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b2e59-a6c2-45d9-ad46-eb5442975e90 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:25.557152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:25.557450 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 420/837] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 19:31:25.561745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed4ddc-fbf0-4655-a9cf-1493b475e400 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:25.567971 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1266e-a1ae-46d5-82f6-f1971d173300 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:25.568369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1266e-a1ae-46d5-82f6-f1971d173300 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:25.568707 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f1266e-a1ae-46d5-82f6-f1971d173300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:25.590836 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-35f1266e-a1ae-46d5-82f6-f1971d173300 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:25.592425 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-35f1266e-a1ae-46d5-82f6-f1971d173300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:25.608053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb27da-f0db-419e-952e-b5780fdc031a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:25.609198 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb27da-f0db-419e-952e-b5780fdc031a None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:25.610322 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb27da-f0db-419e-952e-b5780fdc031a None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:25.610778 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb27da-f0db-419e-952e-b5780fdc031a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:25.623724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb27da-f0db-419e-952e-b5780fdc031a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:25.626484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb27da-f0db-419e-952e-b5780fdc031a None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:25.658840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-92cb27da-f0db-419e-952e-b5780fdc031a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:25.658840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:25.658840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:25.658840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:25.658840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:25.658840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:25.658840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:25.658840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:25.658840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:25.658840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:25.658840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:25.658840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:25.658840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:25.658840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:25.658840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:25.659993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:25.659993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:25.659993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:25.659993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:25.659993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:25.659993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:25.659993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:25.659993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:25.659993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:25.659993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:25.659993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:25.659993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:25.659993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:25.659993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:25.659993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:25.659993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:25.659993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:25.661100 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:25.661100 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:25.661100 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:25.661100 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:25.667615 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 418/838] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:25.669995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb27da-f0db-419e-952e-b5780fdc031a None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:25.701039 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 421/839] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:25 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 19:31:25.720557 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f1266e-a1ae-46d5-82f6-f1971d173300 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:25.813737 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb27da-f0db-419e-952e-b5780fdc031a None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:25.835696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f1266e-a1ae-46d5-82f6-f1971d173300 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:25.853032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:25.866858 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f19cb9-7546-4e34-86ff-cc5a93986165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:25.867551 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f19cb9-7546-4e34-86ff-cc5a93986165 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:25.872596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f19cb9-7546-4e34-86ff-cc5a93986165 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:25.872596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f19cb9-7546-4e34-86ff-cc5a93986165 None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:25.872596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f19cb9-7546-4e34-86ff-cc5a93986165 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:25.872596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f19cb9-7546-4e34-86ff-cc5a93986165 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:25.872596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f19cb9-7546-4e34-86ff-cc5a93986165 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:25.884610 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 419/840] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:31:25.890511 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f19cb9-7546-4e34-86ff-cc5a93986165 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:25.891439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c1078-9322-4d15-be4e-49eb2d37de7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:25.892536 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c1078-9322-4d15-be4e-49eb2d37de7f None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:25.893151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c1078-9322-4d15-be4e-49eb2d37de7f None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:25.893731 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c1078-9322-4d15-be4e-49eb2d37de7f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:25.894701 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c1078-9322-4d15-be4e-49eb2d37de7f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:25.896304 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c1078-9322-4d15-be4e-49eb2d37de7f None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:25.903926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:25.913684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-847c1078-9322-4d15-be4e-49eb2d37de7f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 19:31:25.913684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:25.913684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:31:25.913684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:31:25.913684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:31:25.913684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:31:25.913684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:31:25.913684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:25.913684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 19:31:25.913684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:25.913684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:25.913684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:25.913684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:25.913684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:25.913684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:25.913684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:25.913684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:25.914858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:25.914858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:25.914858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:25.914858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:25.914858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:25.914858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:25.914858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:31:25.914858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:31:25.914858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:31:25.914858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:31:25.914858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:25.914858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:25.914858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:25.914858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:25.914858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:25.914858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:25.915816 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:25.915816 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:25.915816 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:25.915816 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:25.915816 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:25.915816 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:25.915816 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:25.915816 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:25.915816 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:25.915816 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:25.915816 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:31:25.915816 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:25.915816 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 19:31:25.915816 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:25.918454 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 422/841] 10.222.0.13 () {56 vars in 1114 bytes} [Fri May 27 19:31:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:31:25.926559 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c1078-9322-4d15-be4e-49eb2d37de7f None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:26.014197 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f19cb9-7546-4e34-86ff-cc5a93986165 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:26.060506 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c1078-9322-4d15-be4e-49eb2d37de7f None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:26.062058 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463ef18-4733-45ff-b227-5a3c7e19a95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:26.063431 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463ef18-4733-45ff-b227-5a3c7e19a95e None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:26.063720 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463ef18-4733-45ff-b227-5a3c7e19a95e None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:26.064014 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463ef18-4733-45ff-b227-5a3c7e19a95e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:26.072030 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463ef18-4733-45ff-b227-5a3c7e19a95e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:26.073538 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463ef18-4733-45ff-b227-5a3c7e19a95e None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:26.088757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-b463ef18-4733-45ff-b227-5a3c7e19a95e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 19:31:26.088757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:26.088757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:26.088757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:26.088757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:26.088757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:26.088757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:26.088757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:26.088757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:26.088757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:26.088757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:26.088757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:26.088757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:31:26.088757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:31:26.088757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:31:26.088757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:31:26.089825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:26.089825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:26.089825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:31:26.089825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:26.089825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:31:26.089825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:26.089825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:26.089825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:26.089825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:26.089825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:26.089825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:26.089825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:26.089825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:26.089825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:26.089825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:26.089825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:26.089825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:26.089825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:26.090774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:26.090774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:26.090774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:31:26.090774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:31:26.090774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:31:26.090774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:31:26.090774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:31:26.090774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:31:26.090774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 19:31:26.090774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:26.090774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 420/842] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:31:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:31:26.092167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5d78c-c6a3-40a5-96f9-a08e1e42eb66 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:26.092167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5d78c-c6a3-40a5-96f9-a08e1e42eb66 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:26.092167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5d78c-c6a3-40a5-96f9-a08e1e42eb66 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:26.093572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 421/843] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:26.098286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5d78c-c6a3-40a5-96f9-a08e1e42eb66 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:26.103902 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83d7c5-2139-4ee4-85e6-dfe1473f94a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:26.104579 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83d7c5-2139-4ee4-85e6-dfe1473f94a3 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:26.104865 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83d7c5-2139-4ee4-85e6-dfe1473f94a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:26.105164 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83d7c5-2139-4ee4-85e6-dfe1473f94a3 None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:26.105883 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83d7c5-2139-4ee4-85e6-dfe1473f94a3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:26.106187 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83d7c5-2139-4ee4-85e6-dfe1473f94a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:26.108112 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83d7c5-2139-4ee4-85e6-dfe1473f94a3 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:26.117921 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 423/844] 10.222.0.13 () {56 vars in 1114 bytes} [Fri May 27 19:31:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:26.117921 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c2eac-6857-44bd-bbec-abaf31c9cf1e None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:26.117921 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c2eac-6857-44bd-bbec-abaf31c9cf1e None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:26.117921 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319c2eac-6857-44bd-bbec-abaf31c9cf1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:26.135404 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-319c2eac-6857-44bd-bbec-abaf31c9cf1e None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:26.136585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-319c2eac-6857-44bd-bbec-abaf31c9cf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:26.140550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:26.180877 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:26.219558 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 424/845] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:26.220869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c2eac-6857-44bd-bbec-abaf31c9cf1e None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:26.221012 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5d78c-c6a3-40a5-96f9-a08e1e42eb66 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:26.268772 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720dedab-a652-402c-a68f-c9466a27f5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:26.269530 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dedab-a652-402c-a68f-c9466a27f5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:26.269886 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dedab-a652-402c-a68f-c9466a27f5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:26.270173 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720dedab-a652-402c-a68f-c9466a27f5a3 None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:26.271051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720dedab-a652-402c-a68f-c9466a27f5a3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:26.271415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720dedab-a652-402c-a68f-c9466a27f5a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:26.272946 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720dedab-a652-402c-a68f-c9466a27f5a3 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:26.295612 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 422/846] 10.222.0.13 () {56 vars in 1135 bytes} [Fri May 27 19:31:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:26.296504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720dedab-a652-402c-a68f-c9466a27f5a3 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:26.357928 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319c2eac-6857-44bd-bbec-abaf31c9cf1e None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:26.398721 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcee6b7-a100-4ea4-81cb-f3911b6dffe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:26.399790 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcee6b7-a100-4ea4-81cb-f3911b6dffe9 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:26.400351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcee6b7-a100-4ea4-81cb-f3911b6dffe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:26.400842 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcee6b7-a100-4ea4-81cb-f3911b6dffe9 None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:26.401795 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcee6b7-a100-4ea4-81cb-f3911b6dffe9 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:26.402317 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcee6b7-a100-4ea4-81cb-f3911b6dffe9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:26.403805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcee6b7-a100-4ea4-81cb-f3911b6dffe9 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:26.408467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720dedab-a652-402c-a68f-c9466a27f5a3 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:26.409237 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 425/847] 10.222.0.13 () {56 vars in 1114 bytes} [Fri May 27 19:31:26 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:26.410164 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcee6b7-a100-4ea4-81cb-f3911b6dffe9 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:26.417066 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:26.455796 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:26.468074 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cd1c0-199e-4d1f-8c65-95874a26fc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:26.479748 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cd1c0-199e-4d1f-8c65-95874a26fc59 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:26.479748 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cd1c0-199e-4d1f-8c65-95874a26fc59 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:26.479748 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cd1c0-199e-4d1f-8c65-95874a26fc59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:26.487972 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92cd1c0-199e-4d1f-8c65-95874a26fc59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:31:26.498147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-d92cd1c0-199e-4d1f-8c65-95874a26fc59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 19:31:26.498147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:26.498147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:26.498147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:26.498147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:26.498147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:26.498147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:26.498147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:26.498147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:26.498147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:26.498147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:26.498147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:26.498147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:31:26.498147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:31:26.498147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:31:26.498147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:31:26.499186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:31:26.499186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:31:26.499186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:31:26.499186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:31:26.499186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:26.499186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:26.499186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:26.499186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:26.499186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:26.499186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:26.499186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:26.499186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:26.499186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:26.499186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:26.499186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:26.499186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:26.501617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:26.501617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:26.501617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:26.501617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:26.501617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:31:26.501617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:31:26.501617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:31:26.501617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:31:26.501617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:31:26.501617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:31:26.501617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 19:31:26.501617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:26.501617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 423/848] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:31:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:26.503038 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64e167-8779-4852-8376-06f2e9f9de2e None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:26.503329 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64e167-8779-4852-8376-06f2e9f9de2e None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:26.503609 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba64e167-8779-4852-8376-06f2e9f9de2e None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:26.507140 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 424/849] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:26.508098 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba64e167-8779-4852-8376-06f2e9f9de2e None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:26.553424 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcee6b7-a100-4ea4-81cb-f3911b6dffe9 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:26.609878 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18ebc7-5439-407c-9efc-cc58868060d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:26.610627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18ebc7-5439-407c-9efc-cc58868060d1 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:26.610921 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18ebc7-5439-407c-9efc-cc58868060d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:26.611201 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea18ebc7-5439-407c-9efc-cc58868060d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:26.668304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba64e167-8779-4852-8376-06f2e9f9de2e None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:26.704543 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18ebc7-5439-407c-9efc-cc58868060d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:26.705981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0569396-8cfd-42ee-b161-e7dd093573b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:26.706291 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea18ebc7-5439-407c-9efc-cc58868060d1 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:26.706684 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0569396-8cfd-42ee-b161-e7dd093573b1 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:26.706972 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0569396-8cfd-42ee-b161-e7dd093573b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:26.707248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0569396-8cfd-42ee-b161-e7dd093573b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:26.708010 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0569396-8cfd-42ee-b161-e7dd093573b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:26.708317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0569396-8cfd-42ee-b161-e7dd093573b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:26.709797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0569396-8cfd-42ee-b161-e7dd093573b1 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:26.710203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:26.719618 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 425/850] 10.222.0.13 () {56 vars in 1103 bytes} [Fri May 27 19:31:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:26.722713 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5504df-bf90-47cb-9526-8007093156c5 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:26.723200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5504df-bf90-47cb-9526-8007093156c5 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:26.724147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5504df-bf90-47cb-9526-8007093156c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:26.746494 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-7b5504df-bf90-47cb-9526-8007093156c5 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:26.747886 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-7b5504df-bf90-47cb-9526-8007093156c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:26.805769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:26.818117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 426/851] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:31:26 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 19:31:26.818117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18ebc7-5439-407c-9efc-cc58868060d1 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:26.841238 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 426/852] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:26.842001 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5504df-bf90-47cb-9526-8007093156c5 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:26.931262 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea18ebc7-5439-407c-9efc-cc58868060d1 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:26.935010 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:26.960309 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5504df-bf90-47cb-9526-8007093156c5 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:26.969243 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131e449-2682-409f-8262-0be3785b32a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:26.970141 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131e449-2682-409f-8262-0be3785b32a1 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:26.970428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131e449-2682-409f-8262-0be3785b32a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:26.970705 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b131e449-2682-409f-8262-0be3785b32a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:26.981620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b131e449-2682-409f-8262-0be3785b32a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:26.983095 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b131e449-2682-409f-8262-0be3785b32a1 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:26.996688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ee356-8c28-448b-892b-4a0cc70d33e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:26.997508 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ee356-8c28-448b-892b-4a0cc70d33e4 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:26.997790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ee356-8c28-448b-892b-4a0cc70d33e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:26.998060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ee356-8c28-448b-892b-4a0cc70d33e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:27.002789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-b131e449-2682-409f-8262-0be3785b32a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:27.002789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:27.002789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:27.002789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:27.002789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:27.002789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:27.002789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:27.002789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:27.002789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:27.002789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:27.002789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:27.002789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:27.002789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:27.002789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:27.002789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:27.003713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:27.003713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:27.003713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:27.003713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:27.003713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:27.003713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:27.003713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:27.003713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:27.003713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:27.003713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:27.003713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:27.003713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:27.003713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:27.003713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:27.003713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:27.003713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:27.003713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:27.004674 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:27.004674 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:27.004674 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:27.004674 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:27.008644 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 427/853] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:27.009415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131e449-2682-409f-8262-0be3785b32a1 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:27.009722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ee356-8c28-448b-892b-4a0cc70d33e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:27.011202 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ee356-8c28-448b-892b-4a0cc70d33e4 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:27.030912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-236ee356-8c28-448b-892b-4a0cc70d33e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:27.030912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:27.030912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:27.030912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:27.030912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:27.030912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:27.030912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:27.030912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:27.030912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:27.030912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:27.030912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:27.030912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:27.030912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:27.030912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:27.030912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:27.033398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:27.033398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:27.033398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:27.033398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:27.033398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:27.033398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:27.033398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:27.033398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:27.033398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:27.033398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:27.033398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:27.033398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:27.033398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:27.033398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:27.033398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:27.033398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:27.033398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:27.034429 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:27.034429 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:27.034429 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:27.034429 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:27.034429 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 427/854] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:27.034429 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ee356-8c28-448b-892b-4a0cc70d33e4 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:27.069733 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:27.133139 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b131e449-2682-409f-8262-0be3785b32a1 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:27.144990 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ee356-8c28-448b-892b-4a0cc70d33e4 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:27.173740 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1fa5e-d7f6-4e73-a6f0-85ca9850775b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:27.174605 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1fa5e-d7f6-4e73-a6f0-85ca9850775b None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:27.175068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1fa5e-d7f6-4e73-a6f0-85ca9850775b None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:27.175347 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1fa5e-d7f6-4e73-a6f0-85ca9850775b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:27.176109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1fa5e-d7f6-4e73-a6f0-85ca9850775b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:27.178179 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1fa5e-d7f6-4e73-a6f0-85ca9850775b None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:27.188365 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a00735-7684-4cf6-ad55-eab61d3d34ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:27.189276 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a00735-7684-4cf6-ad55-eab61d3d34ca None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:27.189702 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a00735-7684-4cf6-ad55-eab61d3d34ca None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:27.190141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a00735-7684-4cf6-ad55-eab61d3d34ca None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:27.191026 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a00735-7684-4cf6-ad55-eab61d3d34ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:27.191476 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a00735-7684-4cf6-ad55-eab61d3d34ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:27.192433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-33d1fa5e-d7f6-4e73-a6f0-85ca9850775b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 19:31:27.192433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:27.192433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:31:27.192433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:31:27.192433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:31:27.192433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:31:27.192433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:31:27.192433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:27.192433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 19:31:27.192433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:27.192433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:27.192433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:27.192433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:27.192433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:27.192433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:27.192433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:27.193585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:27.193585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:27.193585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:27.193585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:27.193585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:27.193585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:27.193585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:27.193585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:31:27.193585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:31:27.193585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:31:27.193585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:31:27.193585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:27.193585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:27.193585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:27.193585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:27.193585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:27.194606 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:27.194606 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:27.194606 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:27.194606 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:27.194606 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:27.194606 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:27.194606 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:27.194606 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:27.194606 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:27.194606 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:27.194606 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:27.194606 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:31:27.194606 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:27.194606 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 19:31:27.194606 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:27.196224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a00735-7684-4cf6-ad55-eab61d3d34ca None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:27.198606 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 428/855] 10.222.0.13 () {56 vars in 1132 bytes} [Fri May 27 19:31:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:27.199495 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1fa5e-d7f6-4e73-a6f0-85ca9850775b None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:27.210253 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 428/856] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:27.211390 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a00735-7684-4cf6-ad55-eab61d3d34ca None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:27.237339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:27.330942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:27.341137 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a00735-7684-4cf6-ad55-eab61d3d34ca None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:27.354264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1fa5e-d7f6-4e73-a6f0-85ca9850775b None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:27.385098 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ce441-e129-4b44-a65f-75f1f37b62aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:27.386200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ce441-e129-4b44-a65f-75f1f37b62aa None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:27.386683 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ce441-e129-4b44-a65f-75f1f37b62aa None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:27.387161 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70ce441-e129-4b44-a65f-75f1f37b62aa None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:27.388431 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ce441-e129-4b44-a65f-75f1f37b62aa None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:27.388981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ce441-e129-4b44-a65f-75f1f37b62aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:27.391487 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70ce441-e129-4b44-a65f-75f1f37b62aa None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:27.403792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 429/857] 10.222.0.13 () {56 vars in 1126 bytes} [Fri May 27 19:31:27 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:27.403792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ce441-e129-4b44-a65f-75f1f37b62aa None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:27.405689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6403f6-3603-4c5c-9c41-24cd06f03ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:27.406859 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6403f6-3603-4c5c-9c41-24cd06f03ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:27.407142 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6403f6-3603-4c5c-9c41-24cd06f03ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:27.407526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6403f6-3603-4c5c-9c41-24cd06f03ac7 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:27.408553 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6403f6-3603-4c5c-9c41-24cd06f03ac7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:27.408860 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6403f6-3603-4c5c-9c41-24cd06f03ac7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:27.410306 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6403f6-3603-4c5c-9c41-24cd06f03ac7 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:27.428775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 429/858] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:27.428775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6403f6-3603-4c5c-9c41-24cd06f03ac7 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:27.532669 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:27.538744 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70ce441-e129-4b44-a65f-75f1f37b62aa None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:27.554211 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6403f6-3603-4c5c-9c41-24cd06f03ac7 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:27.585136 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0570f1-5599-43c2-b1d4-660e24146336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:27.586144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0570f1-5599-43c2-b1d4-660e24146336 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:27.586488 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0570f1-5599-43c2-b1d4-660e24146336 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:27.587022 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0570f1-5599-43c2-b1d4-660e24146336 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:27.592323 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554175f-55d0-4622-a999-0d6a024c14a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:27.593007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554175f-55d0-4622-a999-0d6a024c14a1 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:27.593795 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554175f-55d0-4622-a999-0d6a024c14a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:27.594056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554175f-55d0-4622-a999-0d6a024c14a1 None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:27.594761 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554175f-55d0-4622-a999-0d6a024c14a1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:27.595148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554175f-55d0-4622-a999-0d6a024c14a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:27.596481 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554175f-55d0-4622-a999-0d6a024c14a1 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:27.596844 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0570f1-5599-43c2-b1d4-660e24146336 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:27.598664 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0570f1-5599-43c2-b1d4-660e24146336 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:27.601452 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 430/859] 10.222.0.13 () {56 vars in 1126 bytes} [Fri May 27 19:31:27 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:27.602412 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554175f-55d0-4622-a999-0d6a024c14a1 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:27.616237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-fa0570f1-5599-43c2-b1d4-660e24146336 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 19:31:27.616237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:27.616237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:27.616237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:27.616237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:27.616237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:27.616237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:27.616237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:27.616237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:27.616237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:27.616237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:27.616237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:27.616237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:31:27.616237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:31:27.616237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:31:27.616237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:31:27.617510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:27.617510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:27.617510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:31:27.617510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:27.617510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:31:27.617510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:27.617510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:27.617510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:27.617510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:27.617510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:27.617510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:27.617510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:27.617510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:27.617510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:27.617510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:27.617510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:27.617510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:27.617510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:27.618837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:27.618837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:27.618837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:31:27.618837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:31:27.618837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:31:27.618837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:31:27.618837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:31:27.618837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:31:27.618837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 19:31:27.618837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:27.622529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 430/860] 10.222.0.13 () {56 vars in 1117 bytes} [Fri May 27 19:31:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:27.623507 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0570f1-5599-43c2-b1d4-660e24146336 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:27.645083 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:27.738060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554175f-55d0-4622-a999-0d6a024c14a1 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:27.744049 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0570f1-5599-43c2-b1d4-660e24146336 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:27.788449 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49290e-5788-48f4-bc7e-299d78524419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:27.789550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb420b9-30aa-4b3e-96ff-5b5c7f63e9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:27.789750 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49290e-5788-48f4-bc7e-299d78524419 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:27.790206 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49290e-5788-48f4-bc7e-299d78524419 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:27.790334 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb420b9-30aa-4b3e-96ff-5b5c7f63e9dd None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:27.790516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb420b9-30aa-4b3e-96ff-5b5c7f63e9dd None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:27.790795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb420b9-30aa-4b3e-96ff-5b5c7f63e9dd None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:27.791040 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49290e-5788-48f4-bc7e-299d78524419 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:27.791662 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb420b9-30aa-4b3e-96ff-5b5c7f63e9dd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:27.791976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb420b9-30aa-4b3e-96ff-5b5c7f63e9dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:27.793861 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb420b9-30aa-4b3e-96ff-5b5c7f63e9dd None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:27.802153 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49290e-5788-48f4-bc7e-299d78524419 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:27.804466 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49290e-5788-48f4-bc7e-299d78524419 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:27.821506 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 431/861] 10.222.0.13 () {56 vars in 1149 bytes} [Fri May 27 19:31:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:27.822942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb420b9-30aa-4b3e-96ff-5b5c7f63e9dd None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:27.823903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-5d49290e-5788-48f4-bc7e-299d78524419 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:27.823903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:27.823903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:27.823903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:27.823903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:27.823903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:27.823903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:27.823903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:27.823903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:27.823903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:27.823903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:27.823903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:27.823903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:31:27.823903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:31:27.823903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:31:27.824765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:31:27.824765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:27.824765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:27.824765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:27.824765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:27.824765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:27.824765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:27.824765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:27.824765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:27.824765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:27.824765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:27.824765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:27.824765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:27.824765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:27.824765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:27.824765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:27.824765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:27.825706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:31:27.825706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:31:27.825706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:31:27.825706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:31:27.825706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:31:27.825706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:31:27.825706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:27.825706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:27.829507 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 431/862] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:27.835371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49290e-5788-48f4-bc7e-299d78524419 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:27.843852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:27.921256 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:27.966385 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb420b9-30aa-4b3e-96ff-5b5c7f63e9dd None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:27.973422 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49290e-5788-48f4-bc7e-299d78524419 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:28.010648 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea1699-316d-425a-9ed2-aa4878913530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:28.011471 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea1699-316d-425a-9ed2-aa4878913530 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:28.011893 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea1699-316d-425a-9ed2-aa4878913530 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:28.012268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea1699-316d-425a-9ed2-aa4878913530 None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:28.013498 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea1699-316d-425a-9ed2-aa4878913530 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:28.013918 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea1699-316d-425a-9ed2-aa4878913530 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:28.014378 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b4b7b-44ab-4794-99a3-05e96d50fe89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:28.015878 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b4b7b-44ab-4794-99a3-05e96d50fe89 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:28.016330 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b4b7b-44ab-4794-99a3-05e96d50fe89 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:28.016574 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea1699-316d-425a-9ed2-aa4878913530 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:28.016893 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b4b7b-44ab-4794-99a3-05e96d50fe89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:28.037688 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 432/863] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:31:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:31:28.039268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea1699-316d-425a-9ed2-aa4878913530 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:28.112196 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b4b7b-44ab-4794-99a3-05e96d50fe89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:28.113637 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b4b7b-44ab-4794-99a3-05e96d50fe89 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:28.130279 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:28.184030 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea1699-316d-425a-9ed2-aa4878913530 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:28.217335 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:28.240106 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b408150-6201-4d68-857e-85e8c6e9fa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:28.240881 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b408150-6201-4d68-857e-85e8c6e9fa5c None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:28.241214 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b408150-6201-4d68-857e-85e8c6e9fa5c None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:28.241766 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b408150-6201-4d68-857e-85e8c6e9fa5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:28.242801 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b408150-6201-4d68-857e-85e8c6e9fa5c None admin] RBAC: Adding query filter params (role.id=6505bbb129404436966ee59aa7a01d8e, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=37bc6d0abb454de19e1ab2f46bf4a593) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:28.243479 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b408150-6201-4d68-857e-85e8c6e9fa5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:28.245654 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b408150-6201-4d68-857e-85e8c6e9fa5c None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:28.252221 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 432/864] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:31:27 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 19:31:28.257505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b4b7b-44ab-4794-99a3-05e96d50fe89 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:28.267150 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 433/865] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:31:28 2022] GET /identity/v3/role_assignments?user.id=37bc6d0abb454de19e1ab2f46bf4a593&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=6505bbb129404436966ee59aa7a01d8e => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:28.385032 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b4b7b-44ab-4794-99a3-05e96d50fe89 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:28.448253 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97fe28-95a4-4e9f-a304-299c2a16ce73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:28.449466 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97fe28-95a4-4e9f-a304-299c2a16ce73 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:28.449989 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97fe28-95a4-4e9f-a304-299c2a16ce73 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:28.450456 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c97fe28-95a4-4e9f-a304-299c2a16ce73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:28.465957 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97fe28-95a4-4e9f-a304-299c2a16ce73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:28.467940 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c97fe28-95a4-4e9f-a304-299c2a16ce73 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:28.489532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-1c97fe28-95a4-4e9f-a304-299c2a16ce73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:28.489532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:28.489532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:28.489532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:28.489532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:28.489532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:28.489532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:28.489532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:28.489532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:28.489532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:28.489532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:28.489532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:28.489532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:28.489532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:28.489532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:28.490504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:28.490504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:28.490504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:28.490504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:28.490504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:28.490504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:28.490504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:28.490504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:28.490504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:28.490504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:28.490504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:28.490504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:28.490504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:28.490504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:28.490504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:28.490504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:28.490504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:28.491475 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:28.491475 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:28.491475 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:28.491475 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:28.493508 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 433/866] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:31:28.499501 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b408150-6201-4d68-857e-85e8c6e9fa5c None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:28.625598 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b408150-6201-4d68-857e-85e8c6e9fa5c None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:28.658857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:28.667614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab26baba-6c37-4dc9-b420-dccaa21bff2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:28.670397 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26baba-6c37-4dc9-b420-dccaa21bff2c None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:28.670397 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26baba-6c37-4dc9-b420-dccaa21bff2c None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:28.670397 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26baba-6c37-4dc9-b420-dccaa21bff2c None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:28.670397 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26baba-6c37-4dc9-b420-dccaa21bff2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:28.670397 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26baba-6c37-4dc9-b420-dccaa21bff2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:28.671891 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab26baba-6c37-4dc9-b420-dccaa21bff2c None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:28.685597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 434/867] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:28.691659 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97fe28-95a4-4e9f-a304-299c2a16ce73 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:28.742378 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:28.804850 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c97fe28-95a4-4e9f-a304-299c2a16ce73 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:28.845253 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310ff9a-34ab-49f6-a208-d85433e3311d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:28.846071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310ff9a-34ab-49f6-a208-d85433e3311d None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:28.846357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310ff9a-34ab-49f6-a208-d85433e3311d None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:28.846635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310ff9a-34ab-49f6-a208-d85433e3311d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:28.854583 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310ff9a-34ab-49f6-a208-d85433e3311d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:28.856068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3310ff9a-34ab-49f6-a208-d85433e3311d None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:28.860224 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a01780-e294-494b-a1f2-b3f474960699 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:28.860627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a01780-e294-494b-a1f2-b3f474960699 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:28.861005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a01780-e294-494b-a1f2-b3f474960699 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:28.863010 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 435/868] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:28.870052 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ad5d3-4c17-4ba8-95c2-27d8d8678e45 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:28.870836 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ad5d3-4c17-4ba8-95c2-27d8d8678e45 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:28.871330 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ad5d3-4c17-4ba8-95c2-27d8d8678e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:28.877555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-3310ff9a-34ab-49f6-a208-d85433e3311d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:28.877555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:28.877555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:28.877555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:28.877555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:28.877555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:28.877555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:28.877555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:28.877555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:28.877555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:28.877555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:28.877555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:28.877555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:31:28.877555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:31:28.877555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:31:28.878529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:31:28.878529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:28.878529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:28.878529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:28.878529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:28.878529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:28.878529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:28.878529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:28.878529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:28.878529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:28.878529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:28.878529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:28.878529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:28.878529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:28.878529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:28.878529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:28.878529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:28.879864 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:31:28.879864 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:31:28.879864 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:31:28.879864 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:31:28.879864 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:31:28.879864 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:31:28.879864 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:28.879864 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:28.879864 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 434/869] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:28.886086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310ff9a-34ab-49f6-a208-d85433e3311d None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:28.898497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-aa9ad5d3-4c17-4ba8-95c2-27d8d8678e45 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:28.900618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-aa9ad5d3-4c17-4ba8-95c2-27d8d8678e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:28.993390 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 436/870] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 19:31:29.008243 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:29.017330 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ad5d3-4c17-4ba8-95c2-27d8d8678e45 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:29.025514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310ff9a-34ab-49f6-a208-d85433e3311d None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:29.072489 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbcbbc-ca15-48c3-aa86-16883ca8e0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:29.073485 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbcbbc-ca15-48c3-aa86-16883ca8e0ed None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:29.074091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbcbbc-ca15-48c3-aa86-16883ca8e0ed None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:29.074515 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbcbbc-ca15-48c3-aa86-16883ca8e0ed None admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:29.075735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bbcbbc-ca15-48c3-aa86-16883ca8e0ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:29.076215 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bbcbbc-ca15-48c3-aa86-16883ca8e0ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:29.077982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bbcbbc-ca15-48c3-aa86-16883ca8e0ed None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:29.094426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 435/871] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:31:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:31:29.100816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbcbbc-ca15-48c3-aa86-16883ca8e0ed None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:29.144486 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ad5d3-4c17-4ba8-95c2-27d8d8678e45 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:29.170201 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:29.194008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45062cec-29f8-43f5-b157-6f84e7678af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:29.194667 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45062cec-29f8-43f5-b157-6f84e7678af8 None admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:29.194953 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45062cec-29f8-43f5-b157-6f84e7678af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:29.195235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45062cec-29f8-43f5-b157-6f84e7678af8 None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:29.196141 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45062cec-29f8-43f5-b157-6f84e7678af8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:29.198377 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45062cec-29f8-43f5-b157-6f84e7678af8 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:29.241273 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbcbbc-ca15-48c3-aa86-16883ca8e0ed None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:29.278754 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847ccff6-b232-4982-80c3-44ef5ea7fda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:29.279323 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ccff6-b232-4982-80c3-44ef5ea7fda6 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:29.279604 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ccff6-b232-4982-80c3-44ef5ea7fda6 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:29.279881 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ccff6-b232-4982-80c3-44ef5ea7fda6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:29.281377 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ccff6-b232-4982-80c3-44ef5ea7fda6 None admin] RBAC: Adding query filter params (role.id=11e7404599ae49afb5edd22dabe1a5b2, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=c700b0adb9f54947a88c1baa86accb9f) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:29.281698 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ccff6-b232-4982-80c3-44ef5ea7fda6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:29.283244 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847ccff6-b232-4982-80c3-44ef5ea7fda6 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:29.290116 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 436/872] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:31:29 2022] GET /identity/v3/role_assignments?user.id=c700b0adb9f54947a88c1baa86accb9f&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=11e7404599ae49afb5edd22dabe1a5b2 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:29.328474 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 437/873] 10.222.0.13 () {60 vars in 1141 bytes} [Fri May 27 19:31:29 2022] POST /identity/v3/services => generated 251 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:31:30.042156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a0a9c-7c0c-4c1d-b078-b03cdb53af74 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:30.042504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a0a9c-7c0c-4c1d-b078-b03cdb53af74 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:30.042728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583a0a9c-7c0c-4c1d-b078-b03cdb53af74 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:30.044370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 437/874] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:30.050699 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe5764-408f-410e-8350-67967af5c0df None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:30.051198 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe5764-408f-410e-8350-67967af5c0df None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:30.051302 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe5764-408f-410e-8350-67967af5c0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:30.069819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-d1fe5764-408f-410e-8350-67967af5c0df None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:30.071049 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d1fe5764-408f-410e-8350-67967af5c0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:30.152999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 438/875] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:30 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:30.191214 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a0a9c-7c0c-4c1d-b078-b03cdb53af74 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:30.206925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:30.315368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583a0a9c-7c0c-4c1d-b078-b03cdb53af74 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:30.353923 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d4cfcc-66b0-4a1a-bc0a-8a491655655b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:30.354609 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4cfcc-66b0-4a1a-bc0a-8a491655655b None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:30.354891 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4cfcc-66b0-4a1a-bc0a-8a491655655b None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:30.355168 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d4cfcc-66b0-4a1a-bc0a-8a491655655b None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:30.355894 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d4cfcc-66b0-4a1a-bc0a-8a491655655b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:30.357375 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d4cfcc-66b0-4a1a-bc0a-8a491655655b None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:30.454531 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 438/876] 10.222.0.13 () {60 vars in 1141 bytes} [Fri May 27 19:31:30 2022] POST /identity/v3/services => generated 269 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:31:30.666269 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96e229-e369-43e2-a0ab-a08b3ae8dd10 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:30.666584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96e229-e369-43e2-a0ab-a08b3ae8dd10 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:30.666753 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d96e229-e369-43e2-a0ab-a08b3ae8dd10 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:30.668381 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 439/877] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:30.674870 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87483f0a-63ff-47de-956d-9a5169017c84 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:30.675169 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87483f0a-63ff-47de-956d-9a5169017c84 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:30.675452 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87483f0a-63ff-47de-956d-9a5169017c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:30.694641 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-87483f0a-63ff-47de-956d-9a5169017c84 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:30.695925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-87483f0a-63ff-47de-956d-9a5169017c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:30.772331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:30.784752 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 439/878] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:30 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:30.807019 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96e229-e369-43e2-a0ab-a08b3ae8dd10 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:30.917036 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d96e229-e369-43e2-a0ab-a08b3ae8dd10 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:30.919576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:30.959469 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5ba37-0cb1-4c72-af5b-adac469b57fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:30.960279 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5ba37-0cb1-4c72-af5b-adac469b57fc None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:30.960558 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5ba37-0cb1-4c72-af5b-adac469b57fc None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:30.960831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5ba37-0cb1-4c72-af5b-adac469b57fc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:30.961527 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5ba37-0cb1-4c72-af5b-adac469b57fc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:30.962895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5ba37-0cb1-4c72-af5b-adac469b57fc None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:30.977460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-3af5ba37-0cb1-4c72-af5b-adac469b57fc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 19:31:30.977460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:30.977460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:31:30.977460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:31:30.977460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:31:30.977460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:31:30.977460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:31:30.977460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:30.977460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 19:31:30.977460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:30.977460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:30.977460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:30.977460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:30.977460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:30.977460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:30.977460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:30.977460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:30.978588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:30.978588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:30.978588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:30.978588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:30.978588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:30.978588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:30.978588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:31:30.978588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:31:30.978588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:31:30.978588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:31:30.978588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:30.978588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:30.978588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:30.978588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:30.978588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:30.978588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:30.979470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:30.979470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:30.979470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:30.979470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:30.979470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:30.979470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:30.979470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:30.979470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:30.979470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:30.979470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:30.979470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:31:30.979470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:30.979470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 19:31:30.979470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:30.979470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 440/879] 10.222.0.13 () {56 vars in 1126 bytes} [Fri May 27 19:31:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:30.985411 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87483f0a-63ff-47de-956d-9a5169017c84 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:31.103329 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87483f0a-63ff-47de-956d-9a5169017c84 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:31.141426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66079f70-6ea9-480a-b951-8ea9303ab61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:31.142099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66079f70-6ea9-480a-b951-8ea9303ab61d None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:31.142380 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66079f70-6ea9-480a-b951-8ea9303ab61d None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:31.142653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66079f70-6ea9-480a-b951-8ea9303ab61d None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:31.143365 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66079f70-6ea9-480a-b951-8ea9303ab61d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:31.143670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66079f70-6ea9-480a-b951-8ea9303ab61d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:31.145265 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66079f70-6ea9-480a-b951-8ea9303ab61d None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:31.151032 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 440/880] 10.222.0.13 () {56 vars in 1122 bytes} [Fri May 27 19:31:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:31.157155 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5ba37-0cb1-4c72-af5b-adac469b57fc None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:31.263234 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5ba37-0cb1-4c72-af5b-adac469b57fc None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:31.299886 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7da7d-8496-407a-90a0-a55fde97e675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:31.300560 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7da7d-8496-407a-90a0-a55fde97e675 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:31.300842 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7da7d-8496-407a-90a0-a55fde97e675 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:31.301149 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7da7d-8496-407a-90a0-a55fde97e675 None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:31.301919 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7da7d-8496-407a-90a0-a55fde97e675 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:31.302228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7da7d-8496-407a-90a0-a55fde97e675 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:31.303599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7da7d-8496-407a-90a0-a55fde97e675 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:31.309312 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 441/881] 10.222.0.13 () {56 vars in 1122 bytes} [Fri May 27 19:31:31 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:31.722216 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226941b-5745-40d0-8d64-ba8f89cb8170 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:31.722935 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226941b-5745-40d0-8d64-ba8f89cb8170 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:31.722935 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226941b-5745-40d0-8d64-ba8f89cb8170 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:31.724344 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 441/882] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:31.729763 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061e997-1548-4919-b4a6-6c1ada466106 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:31.730053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061e997-1548-4919-b4a6-6c1ada466106 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:31.730333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f061e997-1548-4919-b4a6-6c1ada466106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:31.748598 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-f061e997-1548-4919-b4a6-6c1ada466106 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:31.750492 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-f061e997-1548-4919-b4a6-6c1ada466106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:31.779336 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:31.825239 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a2f9d-e6d2-4455-84ea-82b9bd4d810a None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:31.826021 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a2f9d-e6d2-4455-84ea-82b9bd4d810a None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:31.826524 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a2f9d-e6d2-4455-84ea-82b9bd4d810a None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:31.828085 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 442/883] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:31 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:31.829847 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 442/884] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:31.835465 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cda83-f9fc-4ea3-b268-8d09e3749556 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:31.835756 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cda83-f9fc-4ea3-b268-8d09e3749556 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:31.836031 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cda83-f9fc-4ea3-b268-8d09e3749556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:31.854608 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-cf1cda83-f9fc-4ea3-b268-8d09e3749556 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:31.855367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a2f9d-e6d2-4455-84ea-82b9bd4d810a None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:31.855780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-cf1cda83-f9fc-4ea3-b268-8d09e3749556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:31.930653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:31.937764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 443/885] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:31 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:31.962115 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cda83-f9fc-4ea3-b268-8d09e3749556 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:31.987163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a2f9d-e6d2-4455-84ea-82b9bd4d810a None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:32.028595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f2626-2ac4-4418-b6b3-ec40dde61909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:32.029466 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f2626-2ac4-4418-b6b3-ec40dde61909 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:32.029754 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f2626-2ac4-4418-b6b3-ec40dde61909 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:32.030558 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f2626-2ac4-4418-b6b3-ec40dde61909 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:32.031581 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f2626-2ac4-4418-b6b3-ec40dde61909 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:32.033061 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f2626-2ac4-4418-b6b3-ec40dde61909 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:32.046019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-810f2626-2ac4-4418-b6b3-ec40dde61909 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 19:31:32.046019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:32.046019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:31:32.046019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:31:32.046019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:31:32.046019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:31:32.046019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:31:32.046019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:32.046019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 19:31:32.046019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:32.046019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:32.046019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:32.046019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:32.046019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:32.046019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:32.046019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:32.046019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:32.047112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:32.047112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:32.047112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:32.047112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:32.047112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:32.047112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:32.047112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:31:32.047112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:31:32.047112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:31:32.047112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:31:32.047112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:32.047112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:32.047112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:32.047112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:32.047112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:32.047112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:32.047968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:32.047968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:32.047968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:32.047968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:32.047968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:32.047968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:32.047968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:32.047968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:32.047968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:32.047968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:32.047968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:31:32.047968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:32.047968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 19:31:32.047968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:32.052876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 443/886] 10.222.0.13 () {56 vars in 1114 bytes} [Fri May 27 19:31:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:32.054509 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f2626-2ac4-4418-b6b3-ec40dde61909 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:32.098010 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:32.114570 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cda83-f9fc-4ea3-b268-8d09e3749556 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:32.164616 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561f93b-e9c2-4b43-bf99-2b5176f54ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:32.166400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561f93b-e9c2-4b43-bf99-2b5176f54ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:32.166914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561f93b-e9c2-4b43-bf99-2b5176f54ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:32.167441 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561f93b-e9c2-4b43-bf99-2b5176f54ac3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:32.174119 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f2626-2ac4-4418-b6b3-ec40dde61909 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:32.179867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561f93b-e9c2-4b43-bf99-2b5176f54ac3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:31:32.189896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-0561f93b-e9c2-4b43-bf99-2b5176f54ac3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 19:31:32.189896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:32.189896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:32.189896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:32.189896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:32.189896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:32.189896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:32.189896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:32.189896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:32.189896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:32.189896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:32.189896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:32.189896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:31:32.189896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:31:32.189896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:31:32.189896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:31:32.191035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:31:32.191035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:31:32.191035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:31:32.191035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:31:32.191035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:32.191035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:32.191035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:32.191035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:32.191035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:32.191035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:32.191035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:32.191035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:32.191035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:32.191035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:32.191035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:32.191035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:32.191900 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:32.191900 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:32.191900 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:32.191900 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:32.191900 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:31:32.191900 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:31:32.191900 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:31:32.191900 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:31:32.191900 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:31:32.191900 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:31:32.191900 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 19:31:32.191900 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:32.194835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 444/887] 10.222.0.13 () {56 vars in 1120 bytes} [Fri May 27 19:31:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:32.200807 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561f93b-e9c2-4b43-bf99-2b5176f54ac3 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:32.212916 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:32.226373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc158e7b-9e3f-4d76-86b4-fa4bc310972d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:32.229714 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc158e7b-9e3f-4d76-86b4-fa4bc310972d None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:32.229714 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc158e7b-9e3f-4d76-86b4-fa4bc310972d None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:32.229714 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc158e7b-9e3f-4d76-86b4-fa4bc310972d None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:32.229714 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc158e7b-9e3f-4d76-86b4-fa4bc310972d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:32.229714 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc158e7b-9e3f-4d76-86b4-fa4bc310972d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:32.230698 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc158e7b-9e3f-4d76-86b4-fa4bc310972d None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:32.242173 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 444/888] 10.222.0.13 () {56 vars in 1114 bytes} [Fri May 27 19:31:32 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:32.243308 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc158e7b-9e3f-4d76-86b4-fa4bc310972d None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:32.365161 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561f93b-e9c2-4b43-bf99-2b5176f54ac3 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:32.365631 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc158e7b-9e3f-4d76-86b4-fa4bc310972d None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:32.403219 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1506a38-7808-4f19-827f-3dfe258ce8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:32.404369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1506a38-7808-4f19-827f-3dfe258ce8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:32.405503 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1506a38-7808-4f19-827f-3dfe258ce8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:32.406065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1506a38-7808-4f19-827f-3dfe258ce8c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:32.407091 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1506a38-7808-4f19-827f-3dfe258ce8c5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:32.407266 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad99127-7fe4-452a-bed3-61bb26b1db15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:32.407757 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1506a38-7808-4f19-827f-3dfe258ce8c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:32.408104 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad99127-7fe4-452a-bed3-61bb26b1db15 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:32.408462 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad99127-7fe4-452a-bed3-61bb26b1db15 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:32.408820 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad99127-7fe4-452a-bed3-61bb26b1db15 None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:32.409637 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad99127-7fe4-452a-bed3-61bb26b1db15 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:32.410019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad99127-7fe4-452a-bed3-61bb26b1db15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:32.410423 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1506a38-7808-4f19-827f-3dfe258ce8c5 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:32.411576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad99127-7fe4-452a-bed3-61bb26b1db15 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:32.422593 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 445/889] 10.222.0.13 () {56 vars in 1114 bytes} [Fri May 27 19:31:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:32.423867 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad99127-7fe4-452a-bed3-61bb26b1db15 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:32.427048 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 445/890] 10.222.0.13 () {56 vars in 1115 bytes} [Fri May 27 19:31:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:32.432709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1506a38-7808-4f19-827f-3dfe258ce8c5 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:32.449639 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:32.486075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:32.540747 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad99127-7fe4-452a-bed3-61bb26b1db15 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:32.560576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1506a38-7808-4f19-827f-3dfe258ce8c5 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:32.579760 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be36df-ec53-4323-b59b-dd97d7b07f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:32.580427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be36df-ec53-4323-b59b-dd97d7b07f07 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:32.580712 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be36df-ec53-4323-b59b-dd97d7b07f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:32.581013 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be36df-ec53-4323-b59b-dd97d7b07f07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:32.581996 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be36df-ec53-4323-b59b-dd97d7b07f07 None admin] RBAC: Adding query filter params (interface=public, service_id=2aa661bf99cd4ef288bbceca6028f995, region_id=RegionOne) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:32.582233 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be36df-ec53-4323-b59b-dd97d7b07f07 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:32.583619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be36df-ec53-4323-b59b-dd97d7b07f07 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:32.593049 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 446/891] 10.222.0.13 () {56 vars in 1251 bytes} [Fri May 27 19:31:32 2022] GET /identity/v3/endpoints?service_id=2aa661bf99cd4ef288bbceca6028f995&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:32.595525 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be36df-ec53-4323-b59b-dd97d7b07f07 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:32.606034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2f93e-a56e-46ab-95fb-b2fb87344fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:32.606838 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2f93e-a56e-46ab-95fb-b2fb87344fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:32.607119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2f93e-a56e-46ab-95fb-b2fb87344fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:32.607395 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d2f93e-a56e-46ab-95fb-b2fb87344fe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:32.622524 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2f93e-a56e-46ab-95fb-b2fb87344fe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:32.624821 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d2f93e-a56e-46ab-95fb-b2fb87344fe3 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:32.649226 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-b1d2f93e-a56e-46ab-95fb-b2fb87344fe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:32.649226 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:32.649226 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:32.649226 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:32.649226 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:32.649226 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:32.649226 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:32.649226 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:32.649226 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:32.649226 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:32.649226 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:32.649226 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:32.649226 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:32.649226 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:32.649226 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:32.650197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:32.650197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:32.650197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:32.650197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:32.650197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:32.650197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:32.650197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:32.650197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:32.650197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:32.650197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:32.650197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:32.650197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:32.650197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:32.650197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:32.650197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:32.650197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:32.650197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:32.651088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:32.651088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:32.651088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:32.651088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:32.653026 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 446/892] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:31:32.656215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2f93e-a56e-46ab-95fb-b2fb87344fe3 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:32.736525 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be36df-ec53-4323-b59b-dd97d7b07f07 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:32.749936 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:32.766388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:32.789968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a4d61-2a38-4da2-b816-ef7b2d09d9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:32.790720 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a4d61-2a38-4da2-b816-ef7b2d09d9e9 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:32.791009 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a4d61-2a38-4da2-b816-ef7b2d09d9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:32.791285 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a4d61-2a38-4da2-b816-ef7b2d09d9e9 None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:32.792390 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a4d61-2a38-4da2-b816-ef7b2d09d9e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:32.794064 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a4d61-2a38-4da2-b816-ef7b2d09d9e9 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:32.796099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d2f93e-a56e-46ab-95fb-b2fb87344fe3 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:32.802847 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 447/893] 10.222.0.13 () {56 vars in 1087 bytes} [Fri May 27 19:31:32 2022] GET /identity/v3/services => generated 1268 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:31:32.841129 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d804b2f-e519-45b9-a1d3-cdbf597a0bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:32.842782 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d804b2f-e519-45b9-a1d3-cdbf597a0bcf None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:32.843282 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d804b2f-e519-45b9-a1d3-cdbf597a0bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:32.843795 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d804b2f-e519-45b9-a1d3-cdbf597a0bcf None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:32.844984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d804b2f-e519-45b9-a1d3-cdbf597a0bcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:32.845516 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d804b2f-e519-45b9-a1d3-cdbf597a0bcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:32.847139 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d804b2f-e519-45b9-a1d3-cdbf597a0bcf None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:32.863264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 447/894] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:32.869278 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a4d61-2a38-4da2-b816-ef7b2d09d9e9 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:32.936679 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109ffcc-5812-465e-a706-4ccd40a7cb5e None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:32.937646 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109ffcc-5812-465e-a706-4ccd40a7cb5e None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:32.938195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109ffcc-5812-465e-a706-4ccd40a7cb5e None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:32.940969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 448/895] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 19:31:32.946941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386551d-1d99-4bc6-bee7-ad1fcf00c288 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:32.947249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386551d-1d99-4bc6-bee7-ad1fcf00c288 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:32.947513 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386551d-1d99-4bc6-bee7-ad1fcf00c288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:32.972515 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-d386551d-1d99-4bc6-bee7-ad1fcf00c288 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:32.974072 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d386551d-1d99-4bc6-bee7-ad1fcf00c288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:33.000884 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a4d61-2a38-4da2-b816-ef7b2d09d9e9 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:33.043791 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1843c2-7bf3-406f-9083-4a86e5d28f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:33.044518 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1843c2-7bf3-406f-9083-4a86e5d28f89 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:33.044888 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1843c2-7bf3-406f-9083-4a86e5d28f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:33.045199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1843c2-7bf3-406f-9083-4a86e5d28f89 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:33.056232 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1843c2-7bf3-406f-9083-4a86e5d28f89 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:33.057761 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1843c2-7bf3-406f-9083-4a86e5d28f89 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:33.074076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-9e1843c2-7bf3-406f-9083-4a86e5d28f89 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 19:31:33.074076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:33.074076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:33.074076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:33.074076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:33.074076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:33.074076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:33.074076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:33.074076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:33.074076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:33.074076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:33.074076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:33.074076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:31:33.074076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:31:33.074076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:31:33.074076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:31:33.075144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:33.075144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:33.075144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:31:33.075144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:33.075144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:31:33.075144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:33.075144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:33.075144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:33.075144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:33.075144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:33.075144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:33.075144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:33.075144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:33.075144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:33.075144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:33.075144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:33.075144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:33.075144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:33.076088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:33.076088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:33.076088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:31:33.076088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:31:33.076088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:31:33.076088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:31:33.076088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:31:33.076088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:31:33.076088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 19:31:33.076088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:33.076088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 448/896] 10.222.0.13 () {56 vars in 1117 bytes} [Fri May 27 19:31:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:33.082504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1843c2-7bf3-406f-9083-4a86e5d28f89 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:33.089029 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 449/897] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:32 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 19:31:33.112330 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d386551d-1d99-4bc6-bee7-ad1fcf00c288 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:33.142062 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:33.185711 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:33.226174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1843c2-7bf3-406f-9083-4a86e5d28f89 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:33.246452 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d386551d-1d99-4bc6-bee7-ad1fcf00c288 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:33.272632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600153df-4e1b-4a2d-aa68-f8477d62fb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:33.273615 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600153df-4e1b-4a2d-aa68-f8477d62fb2b None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:33.274046 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600153df-4e1b-4a2d-aa68-f8477d62fb2b None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:33.274467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600153df-4e1b-4a2d-aa68-f8477d62fb2b None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:33.275579 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600153df-4e1b-4a2d-aa68-f8477d62fb2b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:33.276062 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600153df-4e1b-4a2d-aa68-f8477d62fb2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:33.277724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600153df-4e1b-4a2d-aa68-f8477d62fb2b None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:33.288164 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035aa99-615b-4992-8412-0bf060c23c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:33.289417 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035aa99-615b-4992-8412-0bf060c23c30 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:33.289641 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035aa99-615b-4992-8412-0bf060c23c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:33.290054 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1035aa99-615b-4992-8412-0bf060c23c30 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:33.290779 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035aa99-615b-4992-8412-0bf060c23c30 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:33.292265 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1035aa99-615b-4992-8412-0bf060c23c30 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:33.303721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 449/898] 10.222.0.13 () {56 vars in 1149 bytes} [Fri May 27 19:31:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:33.304844 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600153df-4e1b-4a2d-aa68-f8477d62fb2b None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:33.306316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-1035aa99-615b-4992-8412-0bf060c23c30 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 19:31:33.306316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:33.306316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:31:33.306316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:31:33.306316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:31:33.306316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:31:33.306316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:31:33.306316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:33.306316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 19:31:33.306316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:33.306316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:33.306316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:33.306316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:33.306316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:33.306316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:33.306316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:33.307264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:33.307264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:33.307264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:33.307264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:33.307264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:33.307264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:33.307264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:33.307264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:31:33.307264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:31:33.307264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:31:33.307264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:31:33.307264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:33.307264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:33.307264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:33.307264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:33.307264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:33.308147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:33.308147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:33.308147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:33.308147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:33.308147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:33.308147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:33.308147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:33.308147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:33.308147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:33.308147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:33.308147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:33.308147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:31:33.308147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:33.308147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 19:31:33.308147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:33.313029 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 450/899] 10.222.0.13 () {56 vars in 1132 bytes} [Fri May 27 19:31:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:31:33.314361 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035aa99-615b-4992-8412-0bf060c23c30 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:33.416370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:33.421468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600153df-4e1b-4a2d-aa68-f8477d62fb2b None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:33.428743 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1035aa99-615b-4992-8412-0bf060c23c30 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:33.452781 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:33.465470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d325ae-7ead-4f8a-9230-434a67cf5800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:33.466481 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d325ae-7ead-4f8a-9230-434a67cf5800 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:33.467567 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d325ae-7ead-4f8a-9230-434a67cf5800 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:33.468166 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d325ae-7ead-4f8a-9230-434a67cf5800 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:33.469547 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec38f6-91fa-4356-bcf1-5dc16dc33850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:33.470221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec38f6-91fa-4356-bcf1-5dc16dc33850 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:33.470505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec38f6-91fa-4356-bcf1-5dc16dc33850 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:33.470782 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec38f6-91fa-4356-bcf1-5dc16dc33850 None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:33.471499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec38f6-91fa-4356-bcf1-5dc16dc33850 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:33.471806 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec38f6-91fa-4356-bcf1-5dc16dc33850 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:33.473255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec38f6-91fa-4356-bcf1-5dc16dc33850 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:33.483559 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 451/900] 10.222.0.13 () {56 vars in 1126 bytes} [Fri May 27 19:31:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:31:33.484847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec38f6-91fa-4356-bcf1-5dc16dc33850 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:33.549041 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d325ae-7ead-4f8a-9230-434a67cf5800 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:33.550918 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d325ae-7ead-4f8a-9230-434a67cf5800 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:33.594522 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec38f6-91fa-4356-bcf1-5dc16dc33850 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:33.631072 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c7ac97-568d-4cdc-821c-6286eaa71eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:33.631751 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7ac97-568d-4cdc-821c-6286eaa71eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:33.632029 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7ac97-568d-4cdc-821c-6286eaa71eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:33.632300 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c7ac97-568d-4cdc-821c-6286eaa71eb7 None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:33.633048 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7ac97-568d-4cdc-821c-6286eaa71eb7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:33.633356 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7ac97-568d-4cdc-821c-6286eaa71eb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:33.634716 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c7ac97-568d-4cdc-821c-6286eaa71eb7 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:33.641021 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 452/901] 10.222.0.13 () {56 vars in 1126 bytes} [Fri May 27 19:31:33 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:33.642408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86b63d-e7a6-4184-804b-1d373679c49c None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:33.642697 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86b63d-e7a6-4184-804b-1d373679c49c None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:33.642972 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe86b63d-e7a6-4184-804b-1d373679c49c None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:33.645268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 453/902] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:33.649029 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86b63d-e7a6-4184-804b-1d373679c49c None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:33.663791 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:33.668720 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 450/903] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:31:33 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 19:31:33.670060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48530d50-d741-4123-8bec-46cd2a54da33 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:33.670887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48530d50-d741-4123-8bec-46cd2a54da33 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:33.671168 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48530d50-d741-4123-8bec-46cd2a54da33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:33.688180 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-48530d50-d741-4123-8bec-46cd2a54da33 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:33.689385 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-48530d50-d741-4123-8bec-46cd2a54da33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:33.709495 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:33.769229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe86b63d-e7a6-4184-804b-1d373679c49c None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:33.769423 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 451/904] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:33 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:33.770023 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48530d50-d741-4123-8bec-46cd2a54da33 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:33.808965 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ae841-a6ee-453a-836f-57472b06dce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:33.809659 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ae841-a6ee-453a-836f-57472b06dce0 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:33.809945 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ae841-a6ee-453a-836f-57472b06dce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:33.810224 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663ae841-a6ee-453a-836f-57472b06dce0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:33.811198 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ae841-a6ee-453a-836f-57472b06dce0 None admin] RBAC: Adding query filter params (interface=public, service_id=d90b20cd8fec4dfa812056fe7405f133, region_id=RegionOne) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:33.811774 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ae841-a6ee-453a-836f-57472b06dce0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:33.813344 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663ae841-a6ee-453a-836f-57472b06dce0 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:33.818992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 454/905] 10.222.0.13 () {56 vars in 1251 bytes} [Fri May 27 19:31:33 2022] GET /identity/v3/endpoints?service_id=d90b20cd8fec4dfa812056fe7405f133&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:33.819895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ae841-a6ee-453a-836f-57472b06dce0 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:33.881774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48530d50-d741-4123-8bec-46cd2a54da33 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:33.915735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:33.925040 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663ae841-a6ee-453a-836f-57472b06dce0 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:33.926135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e889842-81f4-49b5-b979-66ad6f666567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:33.928123 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e889842-81f4-49b5-b979-66ad6f666567 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:33.928426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e889842-81f4-49b5-b979-66ad6f666567 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:33.928708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e889842-81f4-49b5-b979-66ad6f666567 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:33.939021 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:33.941682 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e889842-81f4-49b5-b979-66ad6f666567 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:33.943232 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e889842-81f4-49b5-b979-66ad6f666567 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:33.961525 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a2956-0c00-455f-9ff4-be32df09cddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:33.962198 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a2956-0c00-455f-9ff4-be32df09cddf None admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:33.980546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a2956-0c00-455f-9ff4-be32df09cddf None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:33.980546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a2956-0c00-455f-9ff4-be32df09cddf None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:33.980763 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-6e889842-81f4-49b5-b979-66ad6f666567 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:33.980763 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:33.980763 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:33.980763 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:33.980763 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:33.980763 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:33.980763 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:33.980763 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:33.980763 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:33.980763 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:33.980763 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:33.980763 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:33.980763 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:33.980763 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:33.980763 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:33.981724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:33.981724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:33.981724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:33.981724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:33.981724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:33.981724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:33.981724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:33.981724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:33.981724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:33.981724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:33.981724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:33.981724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:33.981724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:33.981724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:33.981724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:33.981724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:33.981724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:33.982752 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:33.982752 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:33.982752 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:33.982752 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:33.983087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a2956-0c00-455f-9ff4-be32df09cddf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:33.983163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 452/906] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:33.983233 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792a2956-0c00-455f-9ff4-be32df09cddf None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:33.983313 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e889842-81f4-49b5-b979-66ad6f666567 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:34.074503 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e889842-81f4-49b5-b979-66ad6f666567 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:34.109351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 455/907] 10.222.0.13 () {60 vars in 1141 bytes} [Fri May 27 19:31:33 2022] POST /identity/v3/services => generated 241 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:31:34.109351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a2956-0c00-455f-9ff4-be32df09cddf None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:34.131689 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835c926-4671-4489-a609-67d69e3ea6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:34.133039 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835c926-4671-4489-a609-67d69e3ea6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:34.134215 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835c926-4671-4489-a609-67d69e3ea6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:34.134782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835c926-4671-4489-a609-67d69e3ea6a0 None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:34.136057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a835c926-4671-4489-a609-67d69e3ea6a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:34.138474 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a835c926-4671-4489-a609-67d69e3ea6a0 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:34.153538 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 453/908] 10.222.0.13 () {56 vars in 1087 bytes} [Fri May 27 19:31:33 2022] GET /identity/v3/services => generated 1497 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:31:34.244397 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a2956-0c00-455f-9ff4-be32df09cddf None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:34.299671 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80430f-92d0-4d2a-b65a-1a180b7c8b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:34.300420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80430f-92d0-4d2a-b65a-1a180b7c8b86 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:34.300716 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80430f-92d0-4d2a-b65a-1a180b7c8b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:34.301014 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80430f-92d0-4d2a-b65a-1a180b7c8b86 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:34.301770 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80430f-92d0-4d2a-b65a-1a180b7c8b86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:34.302080 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80430f-92d0-4d2a-b65a-1a180b7c8b86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:34.303575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80430f-92d0-4d2a-b65a-1a180b7c8b86 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:34.322627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 456/909] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:34.325957 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835c926-4671-4489-a609-67d69e3ea6a0 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:34.382449 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:34.447224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835c926-4671-4489-a609-67d69e3ea6a0 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:34.485419 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb73ae1-bc39-4e15-a2de-9042c482fcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:34.486487 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb73ae1-bc39-4e15-a2de-9042c482fcc9 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:34.486929 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb73ae1-bc39-4e15-a2de-9042c482fcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:34.487375 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb73ae1-bc39-4e15-a2de-9042c482fcc9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:34.494738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb73ae1-bc39-4e15-a2de-9042c482fcc9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:34.496371 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb73ae1-bc39-4e15-a2de-9042c482fcc9 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:34.513043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-cfb73ae1-bc39-4e15-a2de-9042c482fcc9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:34.513043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:34.513043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:34.513043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:34.513043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:34.513043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:34.513043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:34.513043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:34.513043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:34.513043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:34.513043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:34.513043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:34.513043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:31:34.513043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:31:34.513043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:31:34.514545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:31:34.514545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:34.514545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:34.514545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:34.514545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:34.514545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:34.514545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:34.514545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:34.514545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:34.514545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:34.514545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:34.514545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:34.514545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:34.514545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:34.514545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:34.514545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:34.514545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:34.515734 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:31:34.515734 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:31:34.515734 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:31:34.515734 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:31:34.515734 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:31:34.515734 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:31:34.515734 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:34.515734 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:34.515734 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 454/910] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:34.519658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80430f-92d0-4d2a-b65a-1a180b7c8b86 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:34.556050 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:34.629308 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80430f-92d0-4d2a-b65a-1a180b7c8b86 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:34.666468 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb776f-a596-4bf3-89d1-6853ce0aa1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:34.667141 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb776f-a596-4bf3-89d1-6853ce0aa1bc None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:34.667429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb776f-a596-4bf3-89d1-6853ce0aa1bc None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:34.667712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb776f-a596-4bf3-89d1-6853ce0aa1bc None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:34.668511 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb776f-a596-4bf3-89d1-6853ce0aa1bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:34.668825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb776f-a596-4bf3-89d1-6853ce0aa1bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:34.670871 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bb776f-a596-4bf3-89d1-6853ce0aa1bc None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:34.683714 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 457/911] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:31:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:34.689601 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb73ae1-bc39-4e15-a2de-9042c482fcc9 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:34.796360 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:34.799913 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb73ae1-bc39-4e15-a2de-9042c482fcc9 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:34.836574 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34323be8-83c3-4acb-9a69-90ffe60ac2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:34.837287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34323be8-83c3-4acb-9a69-90ffe60ac2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:34.837780 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34323be8-83c3-4acb-9a69-90ffe60ac2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:34.838056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34323be8-83c3-4acb-9a69-90ffe60ac2d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:34.838885 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34323be8-83c3-4acb-9a69-90ffe60ac2d4 None admin] RBAC: Adding query filter params (role.id=30c027fab0344b47b1ba1ef6c6cbd818, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=c700b0adb9f54947a88c1baa86accb9f) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:34.839197 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34323be8-83c3-4acb-9a69-90ffe60ac2d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:34.840635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34323be8-83c3-4acb-9a69-90ffe60ac2d4 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:34.853104 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 455/912] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:31:34 2022] GET /identity/v3/role_assignments?user.id=c700b0adb9f54947a88c1baa86accb9f&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=30c027fab0344b47b1ba1ef6c6cbd818 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:35.190174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f67fd-3191-49c4-b2cc-823a86812696 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:35.190365 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f67fd-3191-49c4-b2cc-823a86812696 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:35.190675 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f67fd-3191-49c4-b2cc-823a86812696 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:35.192361 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 458/913] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:35.197940 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a5435-a06d-4288-93b1-b7270fca60ef None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:35.198231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a5435-a06d-4288-93b1-b7270fca60ef None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:35.198513 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a5435-a06d-4288-93b1-b7270fca60ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:35.216465 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-417a5435-a06d-4288-93b1-b7270fca60ef None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:35.217764 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-417a5435-a06d-4288-93b1-b7270fca60ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:35.289105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 456/914] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 19:31:35.313089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f67fd-3191-49c4-b2cc-823a86812696 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:35.422833 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:35.432877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f67fd-3191-49c4-b2cc-823a86812696 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:35.470549 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa75200-3f6d-410b-8ab1-68a026db6331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:35.471479 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa75200-3f6d-410b-8ab1-68a026db6331 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:35.472038 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa75200-3f6d-410b-8ab1-68a026db6331 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:35.472319 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa75200-3f6d-410b-8ab1-68a026db6331 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:35.473066 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa75200-3f6d-410b-8ab1-68a026db6331 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:35.474470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa75200-3f6d-410b-8ab1-68a026db6331 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:35.489067 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-daa75200-3f6d-410b-8ab1-68a026db6331 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 19:31:35.489067 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:35.489067 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:31:35.489067 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:31:35.489067 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:31:35.489067 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:31:35.489067 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:31:35.489067 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:35.489067 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 19:31:35.489067 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:35.489067 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:35.489067 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:35.489067 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:35.489067 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:35.489067 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:35.489067 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:35.489067 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:35.491447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:35.491447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:35.491447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:35.491447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:35.491447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:35.491447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:35.491447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:31:35.491447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:31:35.491447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:31:35.491447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:31:35.491447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:35.491447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:35.491447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:35.491447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:35.491447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:35.491447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:35.493550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:35.493550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:35.493550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:35.493550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:35.493550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:35.493550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:35.493550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:35.493550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:35.493550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:35.493550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:35.493550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:31:35.493550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:35.493550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 19:31:35.493550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:35.493550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 459/915] 10.222.0.13 () {56 vars in 1114 bytes} [Fri May 27 19:31:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:35.500996 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417a5435-a06d-4288-93b1-b7270fca60ef None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:35.599887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:35.606810 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417a5435-a06d-4288-93b1-b7270fca60ef None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:35.643162 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54ccf9-6a50-47da-8d61-9eb2e4c6eea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:35.644170 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54ccf9-6a50-47da-8d61-9eb2e4c6eea5 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:35.644647 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54ccf9-6a50-47da-8d61-9eb2e4c6eea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:35.645148 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac54ccf9-6a50-47da-8d61-9eb2e4c6eea5 None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:35.646062 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54ccf9-6a50-47da-8d61-9eb2e4c6eea5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:35.646609 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54ccf9-6a50-47da-8d61-9eb2e4c6eea5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:35.648232 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac54ccf9-6a50-47da-8d61-9eb2e4c6eea5 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:35.653253 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 457/916] 10.222.0.13 () {56 vars in 1114 bytes} [Fri May 27 19:31:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:35.660097 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa75200-3f6d-410b-8ab1-68a026db6331 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:35.771197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa75200-3f6d-410b-8ab1-68a026db6331 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:35.806885 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66162a7-1f6d-4202-9eb2-a7dafdc627ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:35.807859 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66162a7-1f6d-4202-9eb2-a7dafdc627ad None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:35.808357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66162a7-1f6d-4202-9eb2-a7dafdc627ad None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:35.808848 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66162a7-1f6d-4202-9eb2-a7dafdc627ad None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:35.809842 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66162a7-1f6d-4202-9eb2-a7dafdc627ad None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:35.810857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66162a7-1f6d-4202-9eb2-a7dafdc627ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:35.812523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66162a7-1f6d-4202-9eb2-a7dafdc627ad None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:35.818276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 460/917] 10.222.0.13 () {56 vars in 1114 bytes} [Fri May 27 19:31:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:35.824061 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54ccf9-6a50-47da-8d61-9eb2e4c6eea5 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:35.929833 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac54ccf9-6a50-47da-8d61-9eb2e4c6eea5 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:35.966434 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c264e9-84b4-47da-be8c-2d95a2f35a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:35.967314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c264e9-84b4-47da-be8c-2d95a2f35a87 None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:35.967787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c264e9-84b4-47da-be8c-2d95a2f35a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:35.968069 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c264e9-84b4-47da-be8c-2d95a2f35a87 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:35.968828 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c264e9-84b4-47da-be8c-2d95a2f35a87 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:35.970241 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c264e9-84b4-47da-be8c-2d95a2f35a87 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:36.045287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 458/918] 10.222.0.13 () {60 vars in 1144 bytes} [Fri May 27 19:31:35 2022] POST /identity/v3/endpoints => generated 346 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:31:36.505914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b33ed-a8f1-4ace-8f71-9a4d54d8e6a7 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:36.506438 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b33ed-a8f1-4ace-8f71-9a4d54d8e6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:36.506553 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229b33ed-a8f1-4ace-8f71-9a4d54d8e6a7 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:36.508093 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 461/919] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:36.513858 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e8a01-4297-4cc8-899b-734aae069d51 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:36.514156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e8a01-4297-4cc8-899b-734aae069d51 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:36.514433 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390e8a01-4297-4cc8-899b-734aae069d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:36.538734 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-390e8a01-4297-4cc8-899b-734aae069d51 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:36.539909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-390e8a01-4297-4cc8-899b-734aae069d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:36.551752 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:36.572613 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bce07b-6ff7-4826-8ea8-f2ff32ab4be0 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:36.572904 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bce07b-6ff7-4826-8ea8-f2ff32ab4be0 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:36.575488 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bce07b-6ff7-4826-8ea8-f2ff32ab4be0 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:36.575488 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 462/920] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 19:31:36.581370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6affd-1812-4a0a-af98-4e37a5c79c49 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:36.581655 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6affd-1812-4a0a-af98-4e37a5c79c49 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:36.581938 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6affd-1812-4a0a-af98-4e37a5c79c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:36.599824 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-50f6affd-1812-4a0a-af98-4e37a5c79c49 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:36.601445 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-50f6affd-1812-4a0a-af98-4e37a5c79c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:36.617445 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:36.628160 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 459/921] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:36.652933 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e8a01-4297-4cc8-899b-734aae069d51 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:36.680002 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 463/922] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 19:31:36.703745 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6affd-1812-4a0a-af98-4e37a5c79c49 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:36.762893 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390e8a01-4297-4cc8-899b-734aae069d51 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:36.801237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b05a92-7eef-4f0c-8c5c-92235b35130d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:36.802298 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b05a92-7eef-4f0c-8c5c-92235b35130d None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:36.802748 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b05a92-7eef-4f0c-8c5c-92235b35130d None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:36.803184 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b05a92-7eef-4f0c-8c5c-92235b35130d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:36.803991 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b05a92-7eef-4f0c-8c5c-92235b35130d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:36.805560 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b05a92-7eef-4f0c-8c5c-92235b35130d None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:36.809910 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6affd-1812-4a0a-af98-4e37a5c79c49 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:36.819051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-a0b05a92-7eef-4f0c-8c5c-92235b35130d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 19:31:36.819051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:36.819051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:31:36.819051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:31:36.819051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:31:36.819051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:31:36.819051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:31:36.819051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:36.819051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 19:31:36.819051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:36.819051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:36.819051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:36.819051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:36.819051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:36.819051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:36.819051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:36.819051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:36.820304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:36.820304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:36.820304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:36.820304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:36.820304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:36.820304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:36.820304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:31:36.820304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:31:36.820304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:31:36.820304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:31:36.820304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:36.820304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:36.820304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:36.820304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:36.820304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:36.820304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:36.821319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:36.821319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:36.821319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:36.821319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:36.821319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:36.821319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:36.821319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:36.821319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:36.821319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:36.821319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:36.821319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:31:36.821319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:36.821319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 19:31:36.821319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:36.828797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 460/923] 10.222.0.13 () {56 vars in 1126 bytes} [Fri May 27 19:31:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:36.830507 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b05a92-7eef-4f0c-8c5c-92235b35130d None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:36.834581 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:36.849489 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d26d5c2-1caa-4cf6-9a9b-59c191c6b551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:36.858071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26d5c2-1caa-4cf6-9a9b-59c191c6b551 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:36.858071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26d5c2-1caa-4cf6-9a9b-59c191c6b551 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:36.858071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26d5c2-1caa-4cf6-9a9b-59c191c6b551 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:36.858071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d26d5c2-1caa-4cf6-9a9b-59c191c6b551 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:36.858071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d26d5c2-1caa-4cf6-9a9b-59c191c6b551 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:36.867117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-7d26d5c2-1caa-4cf6-9a9b-59c191c6b551 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 19:31:36.867117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:36.867117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:31:36.867117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:31:36.867117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:31:36.867117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:31:36.867117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:31:36.867117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:36.867117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 19:31:36.867117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:36.867117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:36.867117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:36.867117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:36.867117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:36.867117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:36.867117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:36.868311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:36.868311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:36.868311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:36.868311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:36.868311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:36.868311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:36.868311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:36.868311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:31:36.868311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:31:36.868311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:31:36.868311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:31:36.868311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:36.868311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:36.868311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:36.868311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:36.868311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:36.869388 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:36.869388 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:36.869388 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:36.869388 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:36.869388 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:36.869388 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:36.869388 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:36.869388 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:36.869388 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:36.869388 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:36.869388 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:36.869388 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:31:36.869388 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:36.869388 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 19:31:36.869388 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:36.870658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 464/924] 10.222.0.13 () {56 vars in 1132 bytes} [Fri May 27 19:31:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:31:36.882034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d26d5c2-1caa-4cf6-9a9b-59c191c6b551 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:36.901073 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:36.948454 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b05a92-7eef-4f0c-8c5c-92235b35130d None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:36.989119 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887174a4-11a5-4f53-a304-216ed12bfc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:36.989906 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887174a4-11a5-4f53-a304-216ed12bfc4b None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:36.990253 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887174a4-11a5-4f53-a304-216ed12bfc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:36.990609 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887174a4-11a5-4f53-a304-216ed12bfc4b None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:36.991501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887174a4-11a5-4f53-a304-216ed12bfc4b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:36.991886 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887174a4-11a5-4f53-a304-216ed12bfc4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:36.993534 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887174a4-11a5-4f53-a304-216ed12bfc4b None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:36.998512 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d26d5c2-1caa-4cf6-9a9b-59c191c6b551 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:37.003654 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 461/925] 10.222.0.13 () {56 vars in 1122 bytes} [Fri May 27 19:31:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:37.004979 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887174a4-11a5-4f53-a304-216ed12bfc4b None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:37.043829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12aa0f-b97d-4abc-9696-d9c981f9e415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:37.044494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12aa0f-b97d-4abc-9696-d9c981f9e415 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:37.044787 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12aa0f-b97d-4abc-9696-d9c981f9e415 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:37.045121 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12aa0f-b97d-4abc-9696-d9c981f9e415 None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:37.045871 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12aa0f-b97d-4abc-9696-d9c981f9e415 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:37.046520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12aa0f-b97d-4abc-9696-d9c981f9e415 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:37.049116 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12aa0f-b97d-4abc-9696-d9c981f9e415 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:37.060852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 465/926] 10.222.0.13 () {56 vars in 1126 bytes} [Fri May 27 19:31:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:37.062671 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12aa0f-b97d-4abc-9696-d9c981f9e415 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:37.097016 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:37.140844 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887174a4-11a5-4f53-a304-216ed12bfc4b None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:37.165591 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:37.186773 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec375093-54b1-4eae-adcf-9a09ece5d583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:37.187699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec375093-54b1-4eae-adcf-9a09ece5d583 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:37.188165 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec375093-54b1-4eae-adcf-9a09ece5d583 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:37.188646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec375093-54b1-4eae-adcf-9a09ece5d583 None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:37.189565 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec375093-54b1-4eae-adcf-9a09ece5d583 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:37.190065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec375093-54b1-4eae-adcf-9a09ece5d583 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:37.190298 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12aa0f-b97d-4abc-9696-d9c981f9e415 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:37.191914 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec375093-54b1-4eae-adcf-9a09ece5d583 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:37.201948 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 462/927] 10.222.0.13 () {56 vars in 1122 bytes} [Fri May 27 19:31:37 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:37.212101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec375093-54b1-4eae-adcf-9a09ece5d583 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:37.233905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963f0bd-0ceb-4f5c-9357-d0a7e2f00671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:37.234585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963f0bd-0ceb-4f5c-9357-d0a7e2f00671 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:37.234872 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963f0bd-0ceb-4f5c-9357-d0a7e2f00671 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:37.235153 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5963f0bd-0ceb-4f5c-9357-d0a7e2f00671 None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:37.235878 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963f0bd-0ceb-4f5c-9357-d0a7e2f00671 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:37.236200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963f0bd-0ceb-4f5c-9357-d0a7e2f00671 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:37.237638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5963f0bd-0ceb-4f5c-9357-d0a7e2f00671 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:37.243500 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 466/928] 10.222.0.13 () {56 vars in 1126 bytes} [Fri May 27 19:31:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:37.249857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963f0bd-0ceb-4f5c-9357-d0a7e2f00671 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:37.360194 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec375093-54b1-4eae-adcf-9a09ece5d583 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:37.376934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5963f0bd-0ceb-4f5c-9357-d0a7e2f00671 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:37.407535 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:37.413085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a82152a-ee6a-4f2e-a925-ba8c69999076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:37.413748 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82152a-ee6a-4f2e-a925-ba8c69999076 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:37.414034 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82152a-ee6a-4f2e-a925-ba8c69999076 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:37.414315 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82152a-ee6a-4f2e-a925-ba8c69999076 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:37.415115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a82152a-ee6a-4f2e-a925-ba8c69999076 None admin] RBAC: Adding query filter params (interface=public, service_id=3db3f7ea8ca64518988c0100dce0b8e8, region_id=RegionOne) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:37.415429 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a82152a-ee6a-4f2e-a925-ba8c69999076 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:37.416831 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a82152a-ee6a-4f2e-a925-ba8c69999076 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:37.427975 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 463/929] 10.222.0.13 () {56 vars in 1251 bytes} [Fri May 27 19:31:37 2022] GET /identity/v3/endpoints?service_id=3db3f7ea8ca64518988c0100dce0b8e8&interface=public®ion_id=RegionOne => generated 194 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:37.429531 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717a2e9-30fb-4d09-aab4-ab6d07803673 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:37.429889 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717a2e9-30fb-4d09-aab4-ab6d07803673 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:37.430112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6717a2e9-30fb-4d09-aab4-ab6d07803673 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:37.436045 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 464/930] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:37.436045 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717a2e9-30fb-4d09-aab4-ab6d07803673 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:37.442009 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96583b4-5b21-41c9-940a-a1e281ba63d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:37.443611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96583b4-5b21-41c9-940a-a1e281ba63d6 None admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:37.444107 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96583b4-5b21-41c9-940a-a1e281ba63d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:37.444577 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96583b4-5b21-41c9-940a-a1e281ba63d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:37.445516 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96583b4-5b21-41c9-940a-a1e281ba63d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:37.447370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96583b4-5b21-41c9-940a-a1e281ba63d6 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:37.536238 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:37.552247 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 467/931] 10.222.0.13 () {60 vars in 1144 bytes} [Fri May 27 19:31:37 2022] POST /identity/v3/endpoints => generated 347 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:31:37.561732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cddcac7-f5b8-43cc-89bb-7d9cf85fc34b None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:37.562121 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cddcac7-f5b8-43cc-89bb-7d9cf85fc34b None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:37.562405 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cddcac7-f5b8-43cc-89bb-7d9cf85fc34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:37.590652 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-3cddcac7-f5b8-43cc-89bb-7d9cf85fc34b None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:37.593596 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-3cddcac7-f5b8-43cc-89bb-7d9cf85fc34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:37.623444 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6717a2e9-30fb-4d09-aab4-ab6d07803673 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:37.660276 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b333e99-b328-412f-b19e-5aadb6450ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:37.660981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b333e99-b328-412f-b19e-5aadb6450ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:37.661267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b333e99-b328-412f-b19e-5aadb6450ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:37.661548 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b333e99-b328-412f-b19e-5aadb6450ec4 None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:37.662298 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b333e99-b328-412f-b19e-5aadb6450ec4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:37.663713 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b333e99-b328-412f-b19e-5aadb6450ec4 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:37.677414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 465/932] 10.222.0.13 () {56 vars in 1087 bytes} [Fri May 27 19:31:37 2022] GET /identity/v3/services => generated 1497 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:31:37.796118 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 468/933] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:37.826577 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b333e99-b328-412f-b19e-5aadb6450ec4 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:37.925024 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:37.980534 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b333e99-b328-412f-b19e-5aadb6450ec4 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:38.018913 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65631150-aef9-4b48-9cf4-a94edc11fc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:38.020128 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65631150-aef9-4b48-9cf4-a94edc11fc50 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:38.020630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65631150-aef9-4b48-9cf4-a94edc11fc50 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:38.021152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65631150-aef9-4b48-9cf4-a94edc11fc50 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:38.033367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65631150-aef9-4b48-9cf4-a94edc11fc50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:31:38.041986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-65631150-aef9-4b48-9cf4-a94edc11fc50 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 19:31:38.041986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:38.041986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:38.041986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:38.041986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:38.041986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:38.041986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:38.041986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:38.041986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:38.041986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:38.041986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:38.041986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:38.041986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:31:38.041986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:31:38.041986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:31:38.041986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:31:38.042992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:31:38.042992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:31:38.042992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:31:38.042992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:31:38.042992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:38.042992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:38.042992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:38.042992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:38.042992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:38.042992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:38.042992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:38.042992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:38.042992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:38.042992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:38.042992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:38.042992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:38.043843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:38.043843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:38.043843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:38.043843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:38.043843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:31:38.043843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:31:38.043843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:31:38.043843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:31:38.043843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:31:38.043843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:31:38.043843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 19:31:38.043843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:38.046996 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 466/934] 10.222.0.13 () {56 vars in 1120 bytes} [Fri May 27 19:31:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:38.052938 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cddcac7-f5b8-43cc-89bb-7d9cf85fc34b None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:38.161881 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cddcac7-f5b8-43cc-89bb-7d9cf85fc34b None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:38.190894 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:38.198468 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c01ed6-1f9a-4361-85d5-d0f540c0d9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:38.199157 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c01ed6-1f9a-4361-85d5-d0f540c0d9c1 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:38.199444 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c01ed6-1f9a-4361-85d5-d0f540c0d9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:38.199723 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c01ed6-1f9a-4361-85d5-d0f540c0d9c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:38.201081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c01ed6-1f9a-4361-85d5-d0f540c0d9c1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:38.201398 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c01ed6-1f9a-4361-85d5-d0f540c0d9c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:38.202777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c01ed6-1f9a-4361-85d5-d0f540c0d9c1 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:38.212263 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 469/935] 10.222.0.13 () {56 vars in 1115 bytes} [Fri May 27 19:31:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:38.218478 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65631150-aef9-4b48-9cf4-a94edc11fc50 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:38.350196 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65631150-aef9-4b48-9cf4-a94edc11fc50 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:38.381719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:38.389300 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcacee3-a3f0-4309-bbc9-3acdc46e4333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:38.390385 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcacee3-a3f0-4309-bbc9-3acdc46e4333 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:38.390676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcacee3-a3f0-4309-bbc9-3acdc46e4333 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:38.390963 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcacee3-a3f0-4309-bbc9-3acdc46e4333 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:38.402974 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcacee3-a3f0-4309-bbc9-3acdc46e4333 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:38.404983 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcacee3-a3f0-4309-bbc9-3acdc46e4333 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:38.423693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-6fcacee3-a3f0-4309-bbc9-3acdc46e4333 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:38.423693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:38.423693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:38.423693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:38.423693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:38.423693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:38.423693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:38.423693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:38.423693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:38.423693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:38.423693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:38.423693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:38.423693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:38.423693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:38.423693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:38.424692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:38.424692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:38.424692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:38.424692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:38.424692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:38.424692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:38.424692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:38.424692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:38.424692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:38.424692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:38.424692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:38.424692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:38.424692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:38.424692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:38.424692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:38.424692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:38.424692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:38.425693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:38.425693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:38.425693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:38.425693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:38.425693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 467/936] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:31:38.432521 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c01ed6-1f9a-4361-85d5-d0f540c0d9c1 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:38.533291 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d9d73-b498-438b-aa2c-eec3a2bfa73b None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:38.533606 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d9d73-b498-438b-aa2c-eec3a2bfa73b None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:38.534871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d9d73-b498-438b-aa2c-eec3a2bfa73b None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:38.536565 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 468/937] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 19:31:38.540742 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c1be7-c355-4cb7-a80f-f2f5cc06a0d0 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:38.541070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c1be7-c355-4cb7-a80f-f2f5cc06a0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:38.541361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c1be7-c355-4cb7-a80f-f2f5cc06a0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:38.550850 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c01ed6-1f9a-4361-85d5-d0f540c0d9c1 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:38.562963 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-500c1be7-c355-4cb7-a80f-f2f5cc06a0d0 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:38.564572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-500c1be7-c355-4cb7-a80f-f2f5cc06a0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:38.601655 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e310e6-a399-407b-bf86-3d335f6233e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:38.602989 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e310e6-a399-407b-bf86-3d335f6233e4 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:38.603512 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e310e6-a399-407b-bf86-3d335f6233e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:38.604037 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e310e6-a399-407b-bf86-3d335f6233e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:38.605317 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e310e6-a399-407b-bf86-3d335f6233e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:38.606338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e310e6-a399-407b-bf86-3d335f6233e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:38.608326 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e310e6-a399-407b-bf86-3d335f6233e4 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:38.626648 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 470/938] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:38.637065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e310e6-a399-407b-bf86-3d335f6233e4 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:38.669063 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 469/939] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 19:31:38.692225 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c1be7-c355-4cb7-a80f-f2f5cc06a0d0 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:38.761418 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:38.768861 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e310e6-a399-407b-bf86-3d335f6233e4 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:38.821011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6483cd-b622-491b-9168-9abb0dd6399e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:38.821940 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:38.823967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6483cd-b622-491b-9168-9abb0dd6399e None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:38.824531 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6483cd-b622-491b-9168-9abb0dd6399e None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:38.824977 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6483cd-b622-491b-9168-9abb0dd6399e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:38.826791 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c1be7-c355-4cb7-a80f-f2f5cc06a0d0 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:38.836043 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6483cd-b622-491b-9168-9abb0dd6399e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:38.838347 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6483cd-b622-491b-9168-9abb0dd6399e None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:38.862053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-6a6483cd-b622-491b-9168-9abb0dd6399e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 19:31:38.862053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:38.862053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:38.862053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:38.862053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:38.862053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:38.862053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:38.862053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:38.862053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:38.862053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:38.862053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:38.862053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:38.862053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:31:38.862053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:31:38.862053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:31:38.862053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:31:38.863250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:38.863250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:38.863250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:31:38.863250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:38.863250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:31:38.863250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:38.863250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:38.863250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:38.863250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:38.863250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:38.863250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:38.863250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:38.863250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:38.863250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:38.863250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:38.863250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:38.863250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:38.863250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:38.864487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:38.864487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:38.864487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:31:38.864487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:31:38.864487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:31:38.864487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:31:38.864487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:31:38.864487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:31:38.864487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 19:31:38.864487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:38.867221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 471/940] 10.222.0.13 () {56 vars in 1117 bytes} [Fri May 27 19:31:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:38.871695 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ea87b-44f5-4f56-9ef3-8853319645d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:38.872942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6483cd-b622-491b-9168-9abb0dd6399e None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:38.874024 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ea87b-44f5-4f56-9ef3-8853319645d5 None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:38.874305 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ea87b-44f5-4f56-9ef3-8853319645d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:38.874578 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ea87b-44f5-4f56-9ef3-8853319645d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:38.875303 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ea87b-44f5-4f56-9ef3-8853319645d5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:38.877154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ea87b-44f5-4f56-9ef3-8853319645d5 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:38.879600 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: WARNING keystone.server.flask.common [None req-550ea87b-44f5-4f56-9ef3-8853319645d5 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 27 19:31:38.941449 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.sql.core [None req-550ea87b-44f5-4f56-9ef3-8853319645d5 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 19:31:38.941449 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 27 19:31:38.941449 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [parameters: {'id': '3d5433d717a349d3890c2ea54c78fd51', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 27 19:31:38.941449 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80635) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 19:31:38.952715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-550ea87b-44f5-4f56-9ef3-8853319645d5 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3d5433d717a349d3890c2ea54c78fd51.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3d5433d717a349d3890c2ea54c78fd51. May 27 19:31:38.952715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:38.952715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 19:31:38.952715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 19:31:38.952715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 19:31:38.952715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 19:31:38.952715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 19:31:38.952715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application result = self._query(query) May 27 19:31:38.952715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 19:31:38.952715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application conn.query(q) May 27 19:31:38.952715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 19:31:38.952715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 19:31:38.952715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 19:31:38.954085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application result.read() May 27 19:31:38.954085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 19:31:38.954085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 19:31:38.954085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 19:31:38.954085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 19:31:38.954085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 19:31:38.954085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 19:31:38.954085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 19:31:38.954085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 19:31:38.954085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 19:31:38.954085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:38.954085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 19:31:38.954085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:38.954085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:38.954085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 19:31:38.954085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 19:31:38.954085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 27 19:31:38.954085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return project_ref.to_dict() May 27 19:31:38.955125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 19:31:38.955125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application next(self.gen) May 27 19:31:38.955125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 19:31:38.955125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application yield resource May 27 19:31:38.955125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 19:31:38.955125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application next(self.gen) May 27 19:31:38.955125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 19:31:38.955125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self.session.rollback() May 27 19:31:38.955125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 19:31:38.955125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self.force_reraise() May 27 19:31:38.955125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 19:31:38.955125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise self.value May 27 19:31:38.955125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 19:31:38.955125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 19:31:38.955125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 19:31:38.955125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application session.commit() May 27 19:31:38.955125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 19:31:38.956091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 19:31:38.956091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 19:31:38.956091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._prepare_impl() May 27 19:31:38.956091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 19:31:38.956091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self.session.flush() May 27 19:31:38.956091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 19:31:38.956091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._flush(objects) May 27 19:31:38.956091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 19:31:38.956091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 19:31:38.956091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 19:31:38.956091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application compat.raise_( May 27 19:31:38.956091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 19:31:38.956091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception May 27 19:31:38.956091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 19:31:38.956091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application flush_context.execute() May 27 19:31:38.956091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 19:31:38.956091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rec.execute(self) May 27 19:31:38.957219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 19:31:38.957219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 19:31:38.957219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 19:31:38.957219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 19:31:38.957219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 27 19:31:38.957219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application c = connection._execute_20( May 27 19:31:38.957219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 19:31:38.957219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 19:31:38.957219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 19:31:38.957219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 19:31:38.957219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 19:31:38.957219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 19:31:38.957219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 19:31:38.957219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 19:31:38.957219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 19:31:38.959557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 19:31:38.959557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 19:31:38.959557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception May 27 19:31:38.959557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 19:31:38.959557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 19:31:38.959557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 19:31:38.959557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 19:31:38.959557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 19:31:38.959557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application result = self._query(query) May 27 19:31:38.959557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 19:31:38.959557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application conn.query(q) May 27 19:31:38.959557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 19:31:38.959557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 19:31:38.959557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 19:31:38.959557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application result.read() May 27 19:31:38.959557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 19:31:38.959557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 19:31:38.961635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 19:31:38.961635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 19:31:38.961635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 19:31:38.961635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 19:31:38.961635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 19:31:38.961635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 19:31:38.961635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 19:31:38.961635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 27 19:31:38.961635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [parameters: {'id': '3d5433d717a349d3890c2ea54c78fd51', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 27 19:31:38.961635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 19:31:38.961635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:38.961635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:38.961635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:38.961635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:38.961635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 27 19:31:38.963460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 27 19:31:38.963460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 19:31:38.963460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 19:31:38.963460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 27 19:31:38.963460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:38.963460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:38.963460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:38.963460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:38.963460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:38.963460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:38.963460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:38.963460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:38.963460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:38.963460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:38.963460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:38.963460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:38.963460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:38.967437 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:38.967437 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 27 19:31:38.967437 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 27 19:31:38.967437 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:38.967437 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:38.967437 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 27 19:31:38.967437 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.Conflict( May 27 19:31:38.967437 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3d5433d717a349d3890c2ea54c78fd51. May 27 19:31:38.967437 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:38.967437 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 470/941] 10.222.0.13 () {60 vars in 1140 bytes} [Fri May 27 19:31:38 2022] POST /identity/v3/projects => generated 262 bytes in 268 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 27 19:31:38.967437 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ea87b-44f5-4f56-9ef3-8853319645d5 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:39.005590 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6483cd-b622-491b-9168-9abb0dd6399e None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:39.021107 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:39.059568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bdd2e5-1d22-4e28-acbe-21046ce55035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:39.061872 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdd2e5-1d22-4e28-acbe-21046ce55035 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:39.062599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdd2e5-1d22-4e28-acbe-21046ce55035 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:39.063786 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bdd2e5-1d22-4e28-acbe-21046ce55035 None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:39.064977 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bdd2e5-1d22-4e28-acbe-21046ce55035 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:39.066104 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bdd2e5-1d22-4e28-acbe-21046ce55035 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:39.068675 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bdd2e5-1d22-4e28-acbe-21046ce55035 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:39.092909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ea87b-44f5-4f56-9ef3-8853319645d5 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:39.097478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 472/942] 10.222.0.13 () {56 vars in 1149 bytes} [Fri May 27 19:31:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:39.111399 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bdd2e5-1d22-4e28-acbe-21046ce55035 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:39.143112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c46d6-635d-4024-b5b7-4ea1a69093f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:39.144059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c46d6-635d-4024-b5b7-4ea1a69093f0 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:39.144469 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c46d6-635d-4024-b5b7-4ea1a69093f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:39.144818 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c46d6-635d-4024-b5b7-4ea1a69093f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:39.152562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c46d6-635d-4024-b5b7-4ea1a69093f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:39.154439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c46d6-635d-4024-b5b7-4ea1a69093f0 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:39.170795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-d67c46d6-635d-4024-b5b7-4ea1a69093f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:39.170795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:39.170795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:39.170795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:39.170795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:39.170795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:39.170795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:39.170795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:39.170795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:39.170795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:39.170795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:39.170795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:39.170795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:31:39.170795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:31:39.170795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:31:39.171782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:31:39.171782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:39.171782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:39.171782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:39.171782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:39.171782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:39.171782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:39.171782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:39.171782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:39.171782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:39.171782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:39.171782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:39.171782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:39.171782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:39.171782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:39.171782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:39.171782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:39.172694 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:31:39.172694 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:31:39.172694 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:31:39.172694 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:31:39.172694 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:31:39.172694 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:31:39.172694 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:39.172694 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:39.176988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 471/943] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:39.177973 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c46d6-635d-4024-b5b7-4ea1a69093f0 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:39.197238 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:39.257635 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bdd2e5-1d22-4e28-acbe-21046ce55035 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:39.299962 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d4895-8418-400c-a3a6-0d2c953423bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:39.300736 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d4895-8418-400c-a3a6-0d2c953423bf None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:39.300941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d4895-8418-400c-a3a6-0d2c953423bf None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:39.301263 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753d4895-8418-400c-a3a6-0d2c953423bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:39.307009 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c46d6-635d-4024-b5b7-4ea1a69093f0 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:39.313454 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753d4895-8418-400c-a3a6-0d2c953423bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:39.314971 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753d4895-8418-400c-a3a6-0d2c953423bf None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:39.335867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-753d4895-8418-400c-a3a6-0d2c953423bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:39.335867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:39.335867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:39.335867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:39.335867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:39.335867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:39.335867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:39.335867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:39.335867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:39.335867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:39.335867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:39.335867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:39.335867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:39.335867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:39.335867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:39.337248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:39.337248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:39.337248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:39.337248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:39.337248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:39.337248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:39.337248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:39.337248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:39.337248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:39.337248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:39.337248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:39.337248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:39.337248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:39.337248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:39.337248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:39.337248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:39.337248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:39.338173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:39.338173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:39.338173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:39.338173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:39.342307 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 473/944] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:39.343094 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d4895-8418-400c-a3a6-0d2c953423bf None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:39.356415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b35a28-daba-41c0-9f81-57706397be36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:39.358088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b35a28-daba-41c0-9f81-57706397be36 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:39.358593 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b35a28-daba-41c0-9f81-57706397be36 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:39.359349 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b35a28-daba-41c0-9f81-57706397be36 None admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:39.360929 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b35a28-daba-41c0-9f81-57706397be36 None admin] RBAC: Adding query filter params (name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:39.361793 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b35a28-daba-41c0-9f81-57706397be36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:39.364156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b35a28-daba-41c0-9f81-57706397be36 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:39.367732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:39.404068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 472/945] 10.222.0.13 () {56 vars in 1112 bytes} [Fri May 27 19:31:39 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:39.470023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753d4895-8418-400c-a3a6-0d2c953423bf None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:39.513048 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d76415-f227-4b81-9651-4a12794fc04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:39.514387 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d76415-f227-4b81-9651-4a12794fc04e None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:39.515333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d76415-f227-4b81-9651-4a12794fc04e None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:39.515333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d76415-f227-4b81-9651-4a12794fc04e None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:39.515938 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d76415-f227-4b81-9651-4a12794fc04e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:39.516256 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d76415-f227-4b81-9651-4a12794fc04e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:39.517725 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d76415-f227-4b81-9651-4a12794fc04e None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:39.532335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 474/946] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:39.538303 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b35a28-daba-41c0-9f81-57706397be36 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:39.630689 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:39.661075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b35a28-daba-41c0-9f81-57706397be36 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:39.701632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace5471-9040-4e7f-980c-c7670a38e4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:39.702451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace5471-9040-4e7f-980c-c7670a38e4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:39.703080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace5471-9040-4e7f-980c-c7670a38e4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:39.703516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace5471-9040-4e7f-980c-c7670a38e4f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:39.711469 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace5471-9040-4e7f-980c-c7670a38e4f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:39.713234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace5471-9040-4e7f-980c-c7670a38e4f8 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:39.729676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-1ace5471-9040-4e7f-980c-c7670a38e4f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:39.729676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:39.729676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:39.729676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:39.729676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:39.729676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:39.729676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:39.729676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:39.729676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:39.729676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:39.729676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:39.729676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:39.729676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:31:39.729676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:31:39.729676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:31:39.732468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:31:39.732468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:39.732468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:39.732468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:39.732468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:39.732468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:39.732468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:39.732468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:39.732468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:39.732468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:39.732468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:39.732468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:39.732468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:39.732468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:39.732468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:39.732468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:39.732468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:39.733427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:31:39.733427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:31:39.733427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:31:39.733427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:31:39.733427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:31:39.733427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:31:39.733427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:39.733427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:39.733427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 473/947] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:39.738328 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d76415-f227-4b81-9651-4a12794fc04e None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:39.829051 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:39.848841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d76415-f227-4b81-9651-4a12794fc04e None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:39.885459 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac960b83-d6ea-44e8-a21a-529ba3e6fd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:39.886043 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac960b83-d6ea-44e8-a21a-529ba3e6fd04 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:39.886331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac960b83-d6ea-44e8-a21a-529ba3e6fd04 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:39.886609 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac960b83-d6ea-44e8-a21a-529ba3e6fd04 None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:39.887405 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac960b83-d6ea-44e8-a21a-529ba3e6fd04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:39.887779 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac960b83-d6ea-44e8-a21a-529ba3e6fd04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:39.889334 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac960b83-d6ea-44e8-a21a-529ba3e6fd04 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:39.903445 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 475/948] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:31:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:39.910113 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace5471-9040-4e7f-980c-c7670a38e4f8 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:40.027652 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace5471-9040-4e7f-980c-c7670a38e4f8 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:40.065569 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6c536-dfbc-4b59-9e9b-ab43999d64e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:40.066504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6c536-dfbc-4b59-9e9b-ab43999d64e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:40.066787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6c536-dfbc-4b59-9e9b-ab43999d64e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:40.067130 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6c536-dfbc-4b59-9e9b-ab43999d64e9 None admin] PATH_INFO: `/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/c700b0adb9f54947a88c1baa86accb9f/roles/30c027fab0344b47b1ba1ef6c6cbd818` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:40.069705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6c536-dfbc-4b59-9e9b-ab43999d64e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=a47d6b6971c3452e9799e8146856bdc5, user_id=c700b0adb9f54947a88c1baa86accb9f, role_id=30c027fab0344b47b1ba1ef6c6cbd818)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:40.071228 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6c536-dfbc-4b59-9e9b-ab43999d64e9 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:40.073848 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:40.135597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71c82f-9024-4239-9b31-f1a131aa69ab None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:40.135902 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71c82f-9024-4239-9b31-f1a131aa69ab None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:40.136069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef71c82f-9024-4239-9b31-f1a131aa69ab None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:40.137583 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 476/949] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 19:31:40.144175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16d2c9-c9e1-486a-b79b-979aafc84ab6 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:40.144656 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16d2c9-c9e1-486a-b79b-979aafc84ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:40.145132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af16d2c9-c9e1-486a-b79b-979aafc84ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:40.165828 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-af16d2c9-c9e1-486a-b79b-979aafc84ab6 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:40.167047 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-af16d2c9-c9e1-486a-b79b-979aafc84ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:40.180765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 474/950] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:31:39 2022] PUT /identity/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/c700b0adb9f54947a88c1baa86accb9f/roles/30c027fab0344b47b1ba1ef6c6cbd818 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:31:40.269092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 477/951] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 19:31:40.280648 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972d23d-75e7-4e07-a013-a43afc4e7c9f None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:40.280648 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972d23d-75e7-4e07-a013-a43afc4e7c9f None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:40.281192 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972d23d-75e7-4e07-a013-a43afc4e7c9f None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:40.282346 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 475/952] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:40.288543 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e4693-14e8-44d4-927d-33b783f7c74e None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:40.288856 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e4693-14e8-44d4-927d-33b783f7c74e None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:40.289146 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e4693-14e8-44d4-927d-33b783f7c74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:40.293582 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972d23d-75e7-4e07-a013-a43afc4e7c9f None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:40.311845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-dd1e4693-14e8-44d4-927d-33b783f7c74e None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:40.313740 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-dd1e4693-14e8-44d4-927d-33b783f7c74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:40.415042 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:40.416521 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 478/953] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:40.441331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e4693-14e8-44d4-927d-33b783f7c74e None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:40.447948 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972d23d-75e7-4e07-a013-a43afc4e7c9f None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:40.500916 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80112dc-1a96-4696-ac57-b9fbf598e681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:40.501892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80112dc-1a96-4696-ac57-b9fbf598e681 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:40.502181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80112dc-1a96-4696-ac57-b9fbf598e681 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:40.502460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80112dc-1a96-4696-ac57-b9fbf598e681 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:40.503196 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80112dc-1a96-4696-ac57-b9fbf598e681 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:40.505494 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80112dc-1a96-4696-ac57-b9fbf598e681 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:40.522970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-a80112dc-1a96-4696-ac57-b9fbf598e681 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 19:31:40.522970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:40.522970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:31:40.522970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:31:40.522970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:31:40.522970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:31:40.522970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:31:40.522970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:40.522970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 19:31:40.522970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:40.522970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:40.522970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:40.522970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:40.522970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:40.522970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:40.522970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:40.522970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:40.524397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:40.524397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:40.524397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:40.524397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:40.524397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:40.524397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:40.524397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:31:40.524397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:31:40.524397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:31:40.524397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:31:40.524397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:40.524397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:40.524397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:40.524397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:40.524397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:40.524397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:40.525416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:40.525416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:40.525416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:40.525416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:40.525416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:40.525416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:40.525416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:40.525416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:40.525416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:40.525416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:40.525416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:31:40.525416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:40.525416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 19:31:40.525416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:40.526925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 476/954] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:31:40.530977 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80112dc-1a96-4696-ac57-b9fbf598e681 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:40.582706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e4693-14e8-44d4-927d-33b783f7c74e None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:40.602144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:40.624590 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcd153-9016-45d2-8871-3956347ebf74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:40.625722 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcd153-9016-45d2-8871-3956347ebf74 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:40.626155 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcd153-9016-45d2-8871-3956347ebf74 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:40.626703 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcd153-9016-45d2-8871-3956347ebf74 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:40.627604 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adcd153-9016-45d2-8871-3956347ebf74 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:40.629250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adcd153-9016-45d2-8871-3956347ebf74 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:40.643773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-3adcd153-9016-45d2-8871-3956347ebf74 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 19:31:40.643773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:40.643773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:31:40.643773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:31:40.643773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:31:40.643773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:31:40.643773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:31:40.643773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:40.643773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 19:31:40.643773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:40.643773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:40.643773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:40.643773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:40.643773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:40.643773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:40.643773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:40.643773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:40.645138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:40.645138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:40.645138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:40.645138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:40.645138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:40.645138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:40.645138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:31:40.645138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:31:40.645138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:31:40.645138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:31:40.645138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:40.645138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:40.645138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:40.645138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:40.645138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:40.645138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:40.646173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:40.646173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:40.646173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:40.646173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:40.646173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:40.646173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:40.646173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:40.646173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:40.646173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:40.646173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:40.646173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:31:40.646173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:40.646173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 19:31:40.646173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:40.648645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 479/955] 10.222.0.13 () {56 vars in 1126 bytes} [Fri May 27 19:31:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:31:40.656046 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80112dc-1a96-4696-ac57-b9fbf598e681 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:40.657948 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcd153-9016-45d2-8871-3956347ebf74 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:40.701722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158942a3-9906-47a6-9ce3-6847417ef3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:40.702418 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158942a3-9906-47a6-9ce3-6847417ef3ab None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:40.702958 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158942a3-9906-47a6-9ce3-6847417ef3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:40.703238 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158942a3-9906-47a6-9ce3-6847417ef3ab None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:40.703999 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158942a3-9906-47a6-9ce3-6847417ef3ab None admin] RBAC: Adding query filter params (name=compute) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:40.704311 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158942a3-9906-47a6-9ce3-6847417ef3ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:40.706472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158942a3-9906-47a6-9ce3-6847417ef3ab None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:40.720427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 477/956] 10.222.0.13 () {56 vars in 1112 bytes} [Fri May 27 19:31:40 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:31:40.723027 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158942a3-9906-47a6-9ce3-6847417ef3ab None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:40.744730 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:40.807610 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcd153-9016-45d2-8871-3956347ebf74 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:40.855624 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158942a3-9906-47a6-9ce3-6847417ef3ab None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:40.856680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea358f-01eb-4bc6-90f9-0d5b9a1f20de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:40.857393 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea358f-01eb-4bc6-90f9-0d5b9a1f20de None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:40.857675 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea358f-01eb-4bc6-90f9-0d5b9a1f20de None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:40.857950 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea358f-01eb-4bc6-90f9-0d5b9a1f20de None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:40.858698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea358f-01eb-4bc6-90f9-0d5b9a1f20de None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:40.859325 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea358f-01eb-4bc6-90f9-0d5b9a1f20de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:40.861816 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea358f-01eb-4bc6-90f9-0d5b9a1f20de None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:40.870320 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 480/957] 10.222.0.13 () {56 vars in 1122 bytes} [Fri May 27 19:31:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:40.879740 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea358f-01eb-4bc6-90f9-0d5b9a1f20de None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:40.893555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:40.906726 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996f47e-a0e4-4b7e-bf6d-f1c75143ab49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:40.907483 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996f47e-a0e4-4b7e-bf6d-f1c75143ab49 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:40.913957 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996f47e-a0e4-4b7e-bf6d-f1c75143ab49 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:40.913957 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996f47e-a0e4-4b7e-bf6d-f1c75143ab49 None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:40.913957 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f996f47e-a0e4-4b7e-bf6d-f1c75143ab49 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:40.913957 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f996f47e-a0e4-4b7e-bf6d-f1c75143ab49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:40.913957 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f996f47e-a0e4-4b7e-bf6d-f1c75143ab49 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:40.916284 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 478/958] 10.222.0.13 () {56 vars in 1112 bytes} [Fri May 27 19:31:40 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:31:41.015073 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea358f-01eb-4bc6-90f9-0d5b9a1f20de None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:41.051890 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:41.060920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9dcfc3-fc72-4d5f-a7bf-fa49586b175a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:41.061738 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dcfc3-fc72-4d5f-a7bf-fa49586b175a None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:41.062025 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dcfc3-fc72-4d5f-a7bf-fa49586b175a None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:41.062399 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dcfc3-fc72-4d5f-a7bf-fa49586b175a None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:41.063273 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9dcfc3-fc72-4d5f-a7bf-fa49586b175a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:41.063588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9dcfc3-fc72-4d5f-a7bf-fa49586b175a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:41.065320 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9dcfc3-fc72-4d5f-a7bf-fa49586b175a None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:41.071488 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 481/959] 10.222.0.13 () {56 vars in 1122 bytes} [Fri May 27 19:31:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:31:41.078472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996f47e-a0e4-4b7e-bf6d-f1c75143ab49 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:41.201883 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996f47e-a0e4-4b7e-bf6d-f1c75143ab49 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:41.239064 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d219d1b8-30c8-4efd-b097-d0655124ad81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:41.239732 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219d1b8-30c8-4efd-b097-d0655124ad81 None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:41.240002 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219d1b8-30c8-4efd-b097-d0655124ad81 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:41.240272 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d219d1b8-30c8-4efd-b097-d0655124ad81 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:41.241051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d219d1b8-30c8-4efd-b097-d0655124ad81 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:41.242532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d219d1b8-30c8-4efd-b097-d0655124ad81 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:41.342429 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 479/960] 10.222.0.13 () {60 vars in 1144 bytes} [Fri May 27 19:31:41 2022] POST /identity/v3/endpoints => generated 349 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:31:41.933034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f48ff-9cbb-4088-a256-df12d1aa1206 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:41.933584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f48ff-9cbb-4088-a256-df12d1aa1206 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:41.933856 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351f48ff-9cbb-4088-a256-df12d1aa1206 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:41.936683 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 482/961] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:41.946301 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90dbf93-780e-4db0-90b6-6c1eef037d8a None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:41.947166 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90dbf93-780e-4db0-90b6-6c1eef037d8a None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:41.947939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90dbf93-780e-4db0-90b6-6c1eef037d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:41.976392 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-a90dbf93-780e-4db0-90b6-6c1eef037d8a None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:41.979470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-a90dbf93-780e-4db0-90b6-6c1eef037d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:42.039657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:42.113122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 480/962] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:42.150202 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f48ff-9cbb-4088-a256-df12d1aa1206 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:42.297025 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351f48ff-9cbb-4088-a256-df12d1aa1206 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:42.334666 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ac000-6f76-4d24-8b8c-1268f4fbc6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:42.335356 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ac000-6f76-4d24-8b8c-1268f4fbc6cc None admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:42.335646 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ac000-6f76-4d24-8b8c-1268f4fbc6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:42.335929 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ac000-6f76-4d24-8b8c-1268f4fbc6cc None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:42.336640 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ac000-6f76-4d24-8b8c-1268f4fbc6cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:42.338175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ac000-6f76-4d24-8b8c-1268f4fbc6cc None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:42.339508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: WARNING keystone.server.flask.common [None req-f46ac000-6f76-4d24-8b8c-1268f4fbc6cc None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 27 19:31:42.426378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.sql.core [None req-f46ac000-6f76-4d24-8b8c-1268f4fbc6cc None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 19:31:42.426378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 27 19:31:42.426378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [parameters: {'user_id': '3d42e528315245cd8181c08d5a6520df', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 27 19:31:42.426378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80634) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 19:31:42.444505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-f46ac000-6f76-4d24-8b8c-1268f4fbc6cc None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 27 19:31:42.444505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:42.444505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 19:31:42.444505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 19:31:42.444505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 19:31:42.444505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 19:31:42.444505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 19:31:42.444505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application result = self._query(query) May 27 19:31:42.444505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 19:31:42.444505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application conn.query(q) May 27 19:31:42.444505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 19:31:42.444505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 19:31:42.444505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 19:31:42.444505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application result.read() May 27 19:31:42.444505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 19:31:42.445532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 19:31:42.445532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 19:31:42.445532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 19:31:42.445532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 19:31:42.445532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 19:31:42.445532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 19:31:42.445532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 19:31:42.445532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 19:31:42.445532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:42.445532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 19:31:42.445532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:42.445532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:42.445532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 19:31:42.445532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 19:31:42.445532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 27 19:31:42.445532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 27 19:31:42.445532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 19:31:42.445532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application next(self.gen) May 27 19:31:42.446550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 19:31:42.446550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application yield resource May 27 19:31:42.446550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 19:31:42.446550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application next(self.gen) May 27 19:31:42.446550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 19:31:42.446550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self.session.rollback() May 27 19:31:42.446550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 19:31:42.446550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self.force_reraise() May 27 19:31:42.446550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 19:31:42.446550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise self.value May 27 19:31:42.446550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 19:31:42.446550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 19:31:42.446550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 19:31:42.446550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application session.commit() May 27 19:31:42.446550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 19:31:42.446550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 19:31:42.446550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 19:31:42.447499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self._prepare_impl() May 27 19:31:42.447499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 19:31:42.447499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self.session.flush() May 27 19:31:42.447499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 19:31:42.447499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self._flush(objects) May 27 19:31:42.447499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 19:31:42.447499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 19:31:42.447499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 19:31:42.447499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application compat.raise_( May 27 19:31:42.447499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 19:31:42.447499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception May 27 19:31:42.447499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 19:31:42.447499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application flush_context.execute() May 27 19:31:42.447499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 19:31:42.447499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rec.execute(self) May 27 19:31:42.447499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 19:31:42.447499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 19:31:42.448419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 19:31:42.448419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 19:31:42.448419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 27 19:31:42.448419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application result = connection._execute_20( May 27 19:31:42.448419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 19:31:42.448419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 19:31:42.448419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 19:31:42.448419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 19:31:42.448419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 19:31:42.448419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 19:31:42.448419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 19:31:42.448419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 19:31:42.448419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 19:31:42.448419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 19:31:42.448419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 19:31:42.448419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception May 27 19:31:42.449338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 19:31:42.449338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 19:31:42.449338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 19:31:42.449338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 19:31:42.449338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 19:31:42.449338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application result = self._query(query) May 27 19:31:42.449338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 19:31:42.449338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application conn.query(q) May 27 19:31:42.449338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 19:31:42.449338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 19:31:42.449338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 19:31:42.449338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application result.read() May 27 19:31:42.449338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 19:31:42.449338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 19:31:42.449338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 19:31:42.449338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 19:31:42.450831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 19:31:42.450831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 19:31:42.450831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 19:31:42.450831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 19:31:42.450831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 19:31:42.450831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 27 19:31:42.450831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [parameters: {'user_id': '3d42e528315245cd8181c08d5a6520df', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 27 19:31:42.450831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 19:31:42.450831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:42.450831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:42.450831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:42.450831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:42.450831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:42.450831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:42.450831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:42.452852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:42.452852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:42.452852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:42.452852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:42.452852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:42.452852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:42.452852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:42.452852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 27 19:31:42.452852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 27 19:31:42.452852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:42.452852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:42.452852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:31:42.452852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:42.452852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:31:42.452852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:42.452852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 27 19:31:42.452852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 27 19:31:42.460509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 27 19:31:42.460509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 27 19:31:42.460509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 19:31:42.460509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 19:31:42.460509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 27 19:31:42.460509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:42.460509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 483/963] 10.222.0.13 () {60 vars in 1131 bytes} [Fri May 27 19:31:42 2022] POST /identity/v3/users => generated 160 bytes in 297 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:42.461480 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90dbf93-780e-4db0-90b6-6c1eef037d8a None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:42.589829 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90dbf93-780e-4db0-90b6-6c1eef037d8a None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:42.629118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329338c-d73f-474b-bc18-7c76b3b192f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:42.629940 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329338c-d73f-474b-bc18-7c76b3b192f4 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:42.630221 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329338c-d73f-474b-bc18-7c76b3b192f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:42.630497 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329338c-d73f-474b-bc18-7c76b3b192f4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:42.639699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329338c-d73f-474b-bc18-7c76b3b192f4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:42.641225 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329338c-d73f-474b-bc18-7c76b3b192f4 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:42.658153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-d329338c-d73f-474b-bc18-7c76b3b192f4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 19:31:42.658153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:42.658153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:42.658153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:42.658153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:42.658153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:42.658153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:42.658153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:42.658153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:42.658153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:42.658153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:42.658153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:42.658153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:31:42.658153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:31:42.658153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:31:42.658153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:31:42.659204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:42.659204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:42.659204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:31:42.659204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:42.659204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:31:42.659204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:42.659204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:42.659204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:42.659204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:42.659204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:42.659204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:42.659204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:42.659204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:42.659204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:42.659204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:42.659204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:42.659204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:42.659204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:42.660178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:42.660178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:42.660178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:31:42.660178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:31:42.660178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:31:42.660178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:31:42.660178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:31:42.660178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:31:42.660178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 19:31:42.660178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:42.660178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 481/964] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:31:42 2022] GET /identity/v3/users/cinder => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:42.670378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ac000-6f76-4d24-8b8c-1268f4fbc6cc None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:42.769754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:42.823468 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ac000-6f76-4d24-8b8c-1268f4fbc6cc None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:42.865396 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fcdc6-4abd-4fc0-8201-19271d70687f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:42.866001 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fcdc6-4abd-4fc0-8201-19271d70687f None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:42.866281 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fcdc6-4abd-4fc0-8201-19271d70687f None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:42.866557 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39fcdc6-4abd-4fc0-8201-19271d70687f None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:42.867373 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fcdc6-4abd-4fc0-8201-19271d70687f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:42.867678 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fcdc6-4abd-4fc0-8201-19271d70687f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:42.869354 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39fcdc6-4abd-4fc0-8201-19271d70687f None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:42.871581 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3027e-3d56-47dc-99ac-819af868029e None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:42.871893 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3027e-3d56-47dc-99ac-819af868029e None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:42.872205 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3027e-3d56-47dc-99ac-819af868029e None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:42.873684 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 482/965] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:42.879498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4834512-006b-40ae-86c0-878b7c3c168c None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:42.880090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4834512-006b-40ae-86c0-878b7c3c168c None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:42.880610 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4834512-006b-40ae-86c0-878b7c3c168c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:42.908975 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-a4834512-006b-40ae-86c0-878b7c3c168c None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:42.908975 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-a4834512-006b-40ae-86c0-878b7c3c168c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:42.919733 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 484/966] 10.222.0.13 () {56 vars in 1101 bytes} [Fri May 27 19:31:42 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:42.935954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:42.990358 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 483/967] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 19:31:43.014488 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fcdc6-4abd-4fc0-8201-19271d70687f None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:43.135981 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39fcdc6-4abd-4fc0-8201-19271d70687f None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:43.151812 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:43.176130 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3665f-c312-4b52-872c-1ff0cb773003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:43.179778 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3665f-c312-4b52-872c-1ff0cb773003 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:43.179778 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3665f-c312-4b52-872c-1ff0cb773003 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:43.179778 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3665f-c312-4b52-872c-1ff0cb773003 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:43.190030 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3665f-c312-4b52-872c-1ff0cb773003 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:31:43.198884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-b2f3665f-c312-4b52-872c-1ff0cb773003 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 19:31:43.198884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:43.198884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:43.198884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:43.198884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:43.198884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:43.198884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:43.198884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:43.198884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:43.198884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:43.198884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:43.198884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:43.198884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:31:43.198884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:31:43.198884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:31:43.198884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:31:43.199905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:31:43.199905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:31:43.199905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:31:43.199905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:31:43.199905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:43.199905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:43.199905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:43.199905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:43.199905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:43.199905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:43.199905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:43.199905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:43.199905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:43.199905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:43.199905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:43.199905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:43.201020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:43.201020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:43.201020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:43.201020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:43.201020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:31:43.201020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:31:43.201020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:31:43.201020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:31:43.201020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:31:43.201020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:31:43.201020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 19:31:43.201020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:43.201020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 485/968] 10.222.0.13 () {56 vars in 1120 bytes} [Fri May 27 19:31:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:43.208988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4834512-006b-40ae-86c0-878b7c3c168c None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:43.326185 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4834512-006b-40ae-86c0-878b7c3c168c None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:43.372883 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1254c965-068d-4e5e-b966-6a2e37c91fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:43.373604 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254c965-068d-4e5e-b966-6a2e37c91fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:43.373887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254c965-068d-4e5e-b966-6a2e37c91fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:43.374161 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1254c965-068d-4e5e-b966-6a2e37c91fb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:43.374921 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254c965-068d-4e5e-b966-6a2e37c91fb4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:43.375227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254c965-068d-4e5e-b966-6a2e37c91fb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:43.376612 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1254c965-068d-4e5e-b966-6a2e37c91fb4 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:43.394251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 484/969] 10.222.0.13 () {56 vars in 1115 bytes} [Fri May 27 19:31:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:43.395359 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3665f-c312-4b52-872c-1ff0cb773003 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:43.515186 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3665f-c312-4b52-872c-1ff0cb773003 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:43.543562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ba59e-4105-4168-8829-0db717040f0f None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:43.543885 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ba59e-4105-4168-8829-0db717040f0f None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:43.544315 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ba59e-4105-4168-8829-0db717040f0f None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:43.545873 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 485/970] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:43.550894 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e0a6a-a21e-4208-ba58-430a26e29091 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:43.551180 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e0a6a-a21e-4208-ba58-430a26e29091 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:43.551470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e0a6a-a21e-4208-ba58-430a26e29091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:43.563847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4017f3-accf-494e-a9fe-71a3fc1f3bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:43.564802 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4017f3-accf-494e-a9fe-71a3fc1f3bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:43.565128 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4017f3-accf-494e-a9fe-71a3fc1f3bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:43.565422 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4017f3-accf-494e-a9fe-71a3fc1f3bc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:43.571051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-6a4e0a6a-a21e-4208-ba58-430a26e29091 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:43.572349 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-6a4e0a6a-a21e-4208-ba58-430a26e29091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:43.578640 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4017f3-accf-494e-a9fe-71a3fc1f3bc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:43.580147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4017f3-accf-494e-a9fe-71a3fc1f3bc7 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:43.591895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:43.601370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-9c4017f3-accf-494e-a9fe-71a3fc1f3bc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:43.601370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:43.601370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:43.601370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:43.601370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:43.601370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:43.601370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:43.601370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:43.601370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:43.601370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:43.601370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:43.601370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:43.601370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:43.601370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:43.601370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:43.610137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:43.610137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:43.610137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:43.610137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:43.610137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:43.610137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:43.610137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:43.610137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:43.610137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:43.610137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:43.610137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:43.610137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:43.610137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:43.610137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:43.610137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:43.610137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:43.610137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:43.611258 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:43.611258 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:43.611258 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:43.611258 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:43.611258 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 486/971] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:43.611258 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4017f3-accf-494e-a9fe-71a3fc1f3bc7 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:43.640782 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:43.662738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 486/972] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 19:31:43.687371 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e0a6a-a21e-4208-ba58-430a26e29091 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:43.737164 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4017f3-accf-494e-a9fe-71a3fc1f3bc7 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:43.775627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969947c0-9b5b-46b6-b077-d2ef7842341a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:43.776304 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969947c0-9b5b-46b6-b077-d2ef7842341a None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:43.776591 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969947c0-9b5b-46b6-b077-d2ef7842341a None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:43.776871 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969947c0-9b5b-46b6-b077-d2ef7842341a None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:43.778361 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969947c0-9b5b-46b6-b077-d2ef7842341a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:43.778678 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969947c0-9b5b-46b6-b077-d2ef7842341a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:43.780049 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969947c0-9b5b-46b6-b077-d2ef7842341a None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:43.793359 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 487/973] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:31:43.795727 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02c12e-c100-48dd-91db-b3bab1472ec5 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:43.796212 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02c12e-c100-48dd-91db-b3bab1472ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:43.796685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c02c12e-c100-48dd-91db-b3bab1472ec5 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:43.799385 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 488/974] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:43.800639 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02c12e-c100-48dd-91db-b3bab1472ec5 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:43.814924 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e0a6a-a21e-4208-ba58-430a26e29091 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:43.855339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19ec69-138c-4154-a539-d276a896aac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:43.856127 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19ec69-138c-4154-a539-d276a896aac7 None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:43.856411 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19ec69-138c-4154-a539-d276a896aac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:43.856787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19ec69-138c-4154-a539-d276a896aac7 None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:43.857570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e19ec69-138c-4154-a539-d276a896aac7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:43.858973 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e19ec69-138c-4154-a539-d276a896aac7 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:43.896230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:43.913461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c02c12e-c100-48dd-91db-b3bab1472ec5 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:43.923328 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 487/975] 10.222.0.13 () {60 vars in 1141 bytes} [Fri May 27 19:31:43 2022] POST /identity/v3/services => generated 242 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:31:43.924616 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8053dcc-6c3e-43da-8905-b1694f8a5573 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:43.924905 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8053dcc-6c3e-43da-8905-b1694f8a5573 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:43.925212 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8053dcc-6c3e-43da-8905-b1694f8a5573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:43.944775 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-d8053dcc-6c3e-43da-8905-b1694f8a5573 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:43.946134 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d8053dcc-6c3e-43da-8905-b1694f8a5573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:43.950006 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:43.966594 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789d17b-6a1c-4b50-ac0d-9f7f4b107251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:43.967843 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789d17b-6a1c-4b50-ac0d-9f7f4b107251 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:43.968136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789d17b-6a1c-4b50-ac0d-9f7f4b107251 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:43.968416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789d17b-6a1c-4b50-ac0d-9f7f4b107251 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:43.978869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789d17b-6a1c-4b50-ac0d-9f7f4b107251 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:43.980726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789d17b-6a1c-4b50-ac0d-9f7f4b107251 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:43.999210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-4789d17b-6a1c-4b50-ac0d-9f7f4b107251 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 19:31:43.999210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:43.999210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:43.999210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:43.999210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:43.999210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:43.999210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:43.999210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:43.999210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:43.999210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:43.999210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:43.999210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:43.999210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:31:43.999210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:31:43.999210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:31:43.999210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:31:44.000348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:44.000348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:44.000348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:31:44.000348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:44.000348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:31:44.000348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:44.000348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:44.000348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:44.000348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:44.000348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:44.000348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:44.000348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:44.000348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:44.000348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:44.000348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:44.000348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:44.000348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:44.000348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:44.002011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:44.002011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:44.002011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:31:44.002011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:31:44.002011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:31:44.002011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:31:44.002011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:31:44.002011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:31:44.002011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 19:31:44.002011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:44.005318 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 489/976] 10.222.0.13 () {56 vars in 1117 bytes} [Fri May 27 19:31:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:44.006603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789d17b-6a1c-4b50-ac0d-9f7f4b107251 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:44.037920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 488/977] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:43 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:44.065300 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8053dcc-6c3e-43da-8905-b1694f8a5573 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:44.151751 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789d17b-6a1c-4b50-ac0d-9f7f4b107251 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:44.203232 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ced90f-7aa4-48e3-876c-21dc085fd9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:44.204281 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ced90f-7aa4-48e3-876c-21dc085fd9db None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:44.204809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ced90f-7aa4-48e3-876c-21dc085fd9db None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:44.205434 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ced90f-7aa4-48e3-876c-21dc085fd9db None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:44.206954 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ced90f-7aa4-48e3-876c-21dc085fd9db None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:44.207474 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ced90f-7aa4-48e3-876c-21dc085fd9db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:44.209782 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ced90f-7aa4-48e3-876c-21dc085fd9db None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:44.224830 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:44.229100 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8053dcc-6c3e-43da-8905-b1694f8a5573 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:44.239048 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 490/978] 10.222.0.13 () {56 vars in 1149 bytes} [Fri May 27 19:31:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:31:44.244817 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ced90f-7aa4-48e3-876c-21dc085fd9db None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:44.281525 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa5030-b192-4c8b-a80a-239e081e5ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:44.282587 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa5030-b192-4c8b-a80a-239e081e5ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:44.283070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa5030-b192-4c8b-a80a-239e081e5ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:44.283548 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa5030-b192-4c8b-a80a-239e081e5ef2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:44.284478 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa5030-b192-4c8b-a80a-239e081e5ef2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:44.286120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa5030-b192-4c8b-a80a-239e081e5ef2 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:44.300115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-32fa5030-b192-4c8b-a80a-239e081e5ef2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 19:31:44.300115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:44.300115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:31:44.300115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:31:44.300115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:31:44.300115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:31:44.300115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:31:44.300115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:44.300115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 19:31:44.300115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:44.300115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:44.300115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:44.300115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:44.300115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:44.300115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:44.300115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:44.300115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:44.301404 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:44.301404 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:44.301404 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:44.301404 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:44.301404 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:44.301404 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:44.301404 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:31:44.301404 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:31:44.301404 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:31:44.301404 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:31:44.301404 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:44.301404 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:44.301404 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:44.301404 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:44.301404 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:44.301404 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:44.302320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:44.302320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:44.302320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:44.302320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:44.302320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:44.302320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:44.302320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:44.302320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:44.302320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:44.302320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:44.302320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:31:44.302320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:44.302320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 19:31:44.302320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:44.302320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 489/979] 10.222.0.13 () {56 vars in 1126 bytes} [Fri May 27 19:31:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:31:44.307453 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa5030-b192-4c8b-a80a-239e081e5ef2 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:44.312665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:44.364130 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ced90f-7aa4-48e3-876c-21dc085fd9db None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:44.402791 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed269df2-88a4-407e-b95c-da26a0c51c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:44.404027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed269df2-88a4-407e-b95c-da26a0c51c59 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:44.404498 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed269df2-88a4-407e-b95c-da26a0c51c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:44.404789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed269df2-88a4-407e-b95c-da26a0c51c59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:44.422682 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa5030-b192-4c8b-a80a-239e081e5ef2 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:44.458645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247588ef-ed1e-41c2-a845-8706c910f8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:44.459233 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247588ef-ed1e-41c2-a845-8706c910f8fe None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:44.459522 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247588ef-ed1e-41c2-a845-8706c910f8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:44.459800 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247588ef-ed1e-41c2-a845-8706c910f8fe None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:44.460513 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247588ef-ed1e-41c2-a845-8706c910f8fe None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:44.460822 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247588ef-ed1e-41c2-a845-8706c910f8fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:44.462264 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247588ef-ed1e-41c2-a845-8706c910f8fe None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:44.476497 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 490/980] 10.222.0.13 () {56 vars in 1122 bytes} [Fri May 27 19:31:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:31:44.477932 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247588ef-ed1e-41c2-a845-8706c910f8fe None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:44.489786 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:44.494249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed269df2-88a4-407e-b95c-da26a0c51c59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:44.495790 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed269df2-88a4-407e-b95c-da26a0c51c59 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:44.560536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:44.596128 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247588ef-ed1e-41c2-a845-8706c910f8fe None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:44.624109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 491/981] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:31:44 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 19:31:44.625075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed269df2-88a4-407e-b95c-da26a0c51c59 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:44.636611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdddce5-d0cb-4aa1-a5ed-56ab6eadc568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:44.637403 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdddce5-d0cb-4aa1-a5ed-56ab6eadc568 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:44.637683 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdddce5-d0cb-4aa1-a5ed-56ab6eadc568 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:44.638484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdddce5-d0cb-4aa1-a5ed-56ab6eadc568 None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:44.639271 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdddce5-d0cb-4aa1-a5ed-56ab6eadc568 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:44.639575 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdddce5-d0cb-4aa1-a5ed-56ab6eadc568 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:44.641109 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdddce5-d0cb-4aa1-a5ed-56ab6eadc568 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:44.646502 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 491/982] 10.222.0.13 () {56 vars in 1122 bytes} [Fri May 27 19:31:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:31:44.652213 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdddce5-d0cb-4aa1-a5ed-56ab6eadc568 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:44.751471 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:44.756395 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed269df2-88a4-407e-b95c-da26a0c51c59 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:44.774700 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdddce5-d0cb-4aa1-a5ed-56ab6eadc568 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:44.799116 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529bd95c-64c0-4397-b2f6-bd4346c27689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:44.799930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bd95c-64c0-4397-b2f6-bd4346c27689 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:44.800215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bd95c-64c0-4397-b2f6-bd4346c27689 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:44.800494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bd95c-64c0-4397-b2f6-bd4346c27689 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:44.810383 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:44.812964 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529bd95c-64c0-4397-b2f6-bd4346c27689 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:44.814454 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529bd95c-64c0-4397-b2f6-bd4346c27689 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:44.815526 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36359784-fb82-4cb1-bdf2-1eb1a10296c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:44.816383 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36359784-fb82-4cb1-bdf2-1eb1a10296c2 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:44.816729 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36359784-fb82-4cb1-bdf2-1eb1a10296c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:44.817226 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36359784-fb82-4cb1-bdf2-1eb1a10296c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:44.818091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36359784-fb82-4cb1-bdf2-1eb1a10296c2 None admin] RBAC: Adding query filter params (interface=admin, service_id=3db3f7ea8ca64518988c0100dce0b8e8, region_id=RegionOne) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:44.818467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36359784-fb82-4cb1-bdf2-1eb1a10296c2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:44.826698 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36359784-fb82-4cb1-bdf2-1eb1a10296c2 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:44.826698 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 492/983] 10.222.0.13 () {56 vars in 1249 bytes} [Fri May 27 19:31:44 2022] GET /identity/v3/endpoints?service_id=3db3f7ea8ca64518988c0100dce0b8e8&interface=admin®ion_id=RegionOne => generated 193 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:31:44.832115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36359784-fb82-4cb1-bdf2-1eb1a10296c2 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:44.838476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-529bd95c-64c0-4397-b2f6-bd4346c27689 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:44.838476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:44.838476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:44.838476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:44.838476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:44.838476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:44.838476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:44.838476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:44.838476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:44.838476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:44.838476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:44.838476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:44.838476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:44.838476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:44.838476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:44.839934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:44.839934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:44.839934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:44.839934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:44.839934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:44.839934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:44.839934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:44.839934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:44.839934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:44.839934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:44.839934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:44.839934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:44.839934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:44.839934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:44.839934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:44.839934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:44.839934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:44.841510 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:44.841510 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:44.841510 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:44.841510 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:44.841510 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 492/984] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:31:44.847706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529bd95c-64c0-4397-b2f6-bd4346c27689 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:44.940924 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36359784-fb82-4cb1-bdf2-1eb1a10296c2 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:44.977789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529bd95c-64c0-4397-b2f6-bd4346c27689 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:44.978059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0768f91-941b-42fd-8d27-527c9283594b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:44.978547 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0768f91-941b-42fd-8d27-527c9283594b None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:44.978837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0768f91-941b-42fd-8d27-527c9283594b None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:44.979120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0768f91-941b-42fd-8d27-527c9283594b None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:44.979810 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0768f91-941b-42fd-8d27-527c9283594b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:44.981848 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0768f91-941b-42fd-8d27-527c9283594b None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:44.988450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 493/985] 10.222.0.13 () {56 vars in 1087 bytes} [Fri May 27 19:31:44 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:31:45.027809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa222752-6ed7-4917-9e19-3ed69d40ebe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:45.028391 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa222752-6ed7-4917-9e19-3ed69d40ebe3 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:45.028672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa222752-6ed7-4917-9e19-3ed69d40ebe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:45.029007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa222752-6ed7-4917-9e19-3ed69d40ebe3 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:45.029728 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa222752-6ed7-4917-9e19-3ed69d40ebe3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:45.030037 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa222752-6ed7-4917-9e19-3ed69d40ebe3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:45.031463 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa222752-6ed7-4917-9e19-3ed69d40ebe3 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:45.045501 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 493/986] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:31:45.051318 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0768f91-941b-42fd-8d27-527c9283594b None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:45.141666 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:45.192994 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0768f91-941b-42fd-8d27-527c9283594b None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:45.236661 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327907db-063d-46b5-a570-2c7dea9abd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:45.237672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327907db-063d-46b5-a570-2c7dea9abd14 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:45.238053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327907db-063d-46b5-a570-2c7dea9abd14 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:45.238432 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327907db-063d-46b5-a570-2c7dea9abd14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:45.246181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327907db-063d-46b5-a570-2c7dea9abd14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:45.247470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327907db-063d-46b5-a570-2c7dea9abd14 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:45.262646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-327907db-063d-46b5-a570-2c7dea9abd14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:45.262646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:45.262646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:45.262646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:45.262646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:45.262646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:45.262646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:45.262646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:45.262646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:45.262646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:45.262646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:45.262646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:45.262646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:31:45.262646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:31:45.262646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:31:45.263995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:31:45.263995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:45.263995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:45.263995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:45.263995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:45.263995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:45.263995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:45.263995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:45.263995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:45.263995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:45.263995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:45.263995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:45.263995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:45.263995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:45.263995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:45.263995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:45.263995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:45.265147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:31:45.265147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:31:45.265147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:31:45.265147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:31:45.265147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:31:45.265147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:31:45.265147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:45.265147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:45.266506 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 494/987] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:45.273414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa222752-6ed7-4917-9e19-3ed69d40ebe3 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:45.293093 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:45.397495 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa222752-6ed7-4917-9e19-3ed69d40ebe3 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:45.436829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec5153-01ac-4f57-95d9-52f31d93a1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:45.437787 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec5153-01ac-4f57-95d9-52f31d93a1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:45.438310 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec5153-01ac-4f57-95d9-52f31d93a1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:45.438760 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec5153-01ac-4f57-95d9-52f31d93a1a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:45.439792 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec5153-01ac-4f57-95d9-52f31d93a1a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:45.440260 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec5153-01ac-4f57-95d9-52f31d93a1a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:45.441931 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec5153-01ac-4f57-95d9-52f31d93a1a3 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:45.460993 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 494/988] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:31:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:45.462621 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327907db-063d-46b5-a570-2c7dea9abd14 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:45.587788 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327907db-063d-46b5-a570-2c7dea9abd14 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:45.609053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:45.624009 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685cd3dd-3160-41ec-9845-e37034822d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:45.629776 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cd3dd-3160-41ec-9845-e37034822d87 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:45.629776 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cd3dd-3160-41ec-9845-e37034822d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:45.629776 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cd3dd-3160-41ec-9845-e37034822d87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:45.629776 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685cd3dd-3160-41ec-9845-e37034822d87 None admin] RBAC: Adding query filter params (role.id=30c027fab0344b47b1ba1ef6c6cbd818, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=c700b0adb9f54947a88c1baa86accb9f) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:45.629776 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685cd3dd-3160-41ec-9845-e37034822d87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:45.629776 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685cd3dd-3160-41ec-9845-e37034822d87 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:45.639438 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 495/989] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:31:45 2022] GET /identity/v3/role_assignments?user.id=c700b0adb9f54947a88c1baa86accb9f&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=30c027fab0344b47b1ba1ef6c6cbd818 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:46.237697 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365322a-66dc-433f-b79c-d150529f552c None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:46.238076 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365322a-66dc-433f-b79c-d150529f552c None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:46.238374 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365322a-66dc-433f-b79c-d150529f552c None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:46.240114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 495/990] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:46.245832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270f96a-8959-4a5e-a56c-a45da9ef51dd None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:46.246337 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270f96a-8959-4a5e-a56c-a45da9ef51dd None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:46.246415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270f96a-8959-4a5e-a56c-a45da9ef51dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:46.264407 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-5270f96a-8959-4a5e-a56c-a45da9ef51dd None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:46.265649 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-5270f96a-8959-4a5e-a56c-a45da9ef51dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:46.344257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 496/991] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:46 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:46.368499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365322a-66dc-433f-b79c-d150529f552c None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:46.447996 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:46.482947 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365322a-66dc-433f-b79c-d150529f552c None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:46.528565 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814eb42-c58d-4f32-ae4c-e6b80bd1f389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:46.529427 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814eb42-c58d-4f32-ae4c-e6b80bd1f389 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:46.529715 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814eb42-c58d-4f32-ae4c-e6b80bd1f389 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:46.530067 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814eb42-c58d-4f32-ae4c-e6b80bd1f389 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:46.530714 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1814eb42-c58d-4f32-ae4c-e6b80bd1f389 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:46.532097 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1814eb42-c58d-4f32-ae4c-e6b80bd1f389 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:46.545195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-1814eb42-c58d-4f32-ae4c-e6b80bd1f389 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 19:31:46.545195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:46.545195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:31:46.545195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:31:46.545195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:31:46.545195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:31:46.545195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:31:46.545195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:46.545195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 19:31:46.545195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:46.545195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:46.545195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:46.545195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:46.545195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:46.545195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:46.545195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:46.545195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:46.546618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:46.546618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:46.546618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:46.546618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:46.546618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:46.546618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:46.546618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:31:46.546618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:31:46.546618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:31:46.546618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:31:46.546618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:46.546618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:46.546618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:46.546618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:46.546618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:46.546618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:46.548364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:46.548364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:46.548364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:46.548364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:46.548364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:46.548364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:46.548364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:46.548364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:46.548364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:46.548364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:46.548364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:31:46.548364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:46.548364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 19:31:46.548364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:46.548364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 496/992] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:46.560421 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270f96a-8959-4a5e-a56c-a45da9ef51dd None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:46.676978 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270f96a-8959-4a5e-a56c-a45da9ef51dd None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:46.693348 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:46.714457 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acdc11-4bbf-41f0-95e5-aad3fdf911c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:46.715142 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acdc11-4bbf-41f0-95e5-aad3fdf911c1 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:46.715426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acdc11-4bbf-41f0-95e5-aad3fdf911c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:46.715704 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7acdc11-4bbf-41f0-95e5-aad3fdf911c1 None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:46.716418 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7acdc11-4bbf-41f0-95e5-aad3fdf911c1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:46.716726 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7acdc11-4bbf-41f0-95e5-aad3fdf911c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:46.718161 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7acdc11-4bbf-41f0-95e5-aad3fdf911c1 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:46.723933 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 497/993] 10.222.0.13 () {56 vars in 1112 bytes} [Fri May 27 19:31:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:31:46.729490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814eb42-c58d-4f32-ae4c-e6b80bd1f389 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:46.848479 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814eb42-c58d-4f32-ae4c-e6b80bd1f389 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:46.884572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2a0f3-0e53-43bf-b8a8-67deb50382d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:46.885286 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2a0f3-0e53-43bf-b8a8-67deb50382d7 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:46.885573 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2a0f3-0e53-43bf-b8a8-67deb50382d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:46.885849 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2a0f3-0e53-43bf-b8a8-67deb50382d7 None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:46.887243 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2a0f3-0e53-43bf-b8a8-67deb50382d7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:46.887558 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2a0f3-0e53-43bf-b8a8-67deb50382d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:46.888965 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb2a0f3-0e53-43bf-b8a8-67deb50382d7 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:46.890598 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:46.900238 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 497/994] 10.222.0.13 () {56 vars in 1112 bytes} [Fri May 27 19:31:46 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:46.902102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acdc11-4bbf-41f0-95e5-aad3fdf911c1 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:47.009690 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7acdc11-4bbf-41f0-95e5-aad3fdf911c1 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:47.049114 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d153ddea-467c-4949-926b-23bdd7d86879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:47.049114 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153ddea-467c-4949-926b-23bdd7d86879 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:47.049114 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153ddea-467c-4949-926b-23bdd7d86879 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:47.049114 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d153ddea-467c-4949-926b-23bdd7d86879 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:47.049769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153ddea-467c-4949-926b-23bdd7d86879 None admin] RBAC: Adding query filter params (interface=public, service_id=972139a74be943239ca9d4ed2f57fb2b, region_id=RegionOne) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:47.050036 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153ddea-467c-4949-926b-23bdd7d86879 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:47.051282 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d153ddea-467c-4949-926b-23bdd7d86879 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:47.062037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 498/995] 10.222.0.13 () {56 vars in 1251 bytes} [Fri May 27 19:31:46 2022] GET /identity/v3/endpoints?service_id=972139a74be943239ca9d4ed2f57fb2b&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:31:47.064478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2a0f3-0e53-43bf-b8a8-67deb50382d7 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:47.179927 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2a0f3-0e53-43bf-b8a8-67deb50382d7 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:47.217093 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b57b241-fde0-4d17-b991-46d0abc1ee5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:47.217770 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57b241-fde0-4d17-b991-46d0abc1ee5c None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:47.218054 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57b241-fde0-4d17-b991-46d0abc1ee5c None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:47.218332 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b57b241-fde0-4d17-b991-46d0abc1ee5c None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:47.219019 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57b241-fde0-4d17-b991-46d0abc1ee5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:47.220386 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b57b241-fde0-4d17-b991-46d0abc1ee5c None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:47.227102 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 498/996] 10.222.0.13 () {56 vars in 1087 bytes} [Fri May 27 19:31:47 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:31:47.316523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d29d2-243e-42f1-85c0-b41b24d09e12 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:47.317079 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d29d2-243e-42f1-85c0-b41b24d09e12 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:47.317160 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d29d2-243e-42f1-85c0-b41b24d09e12 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:47.318722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 499/997] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:47.325009 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb74602-8d47-4d44-b8d0-e9f85d71f496 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:47.325208 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb74602-8d47-4d44-b8d0-e9f85d71f496 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:47.325419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb74602-8d47-4d44-b8d0-e9f85d71f496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:47.344082 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-6bb74602-8d47-4d44-b8d0-e9f85d71f496 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:47.345677 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-6bb74602-8d47-4d44-b8d0-e9f85d71f496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:47.417803 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:47.434631 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 499/998] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:47.456534 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d29d2-243e-42f1-85c0-b41b24d09e12 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:47.512333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:47.577105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d29d2-243e-42f1-85c0-b41b24d09e12 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:47.619865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5974d-432f-4366-9a20-fc62e1eb0c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:47.620732 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5974d-432f-4366-9a20-fc62e1eb0c7b None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:47.621042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5974d-432f-4366-9a20-fc62e1eb0c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:47.621327 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5974d-432f-4366-9a20-fc62e1eb0c7b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:47.621983 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5974d-432f-4366-9a20-fc62e1eb0c7b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:47.623415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5974d-432f-4366-9a20-fc62e1eb0c7b None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:47.636691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-a4b5974d-432f-4366-9a20-fc62e1eb0c7b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 19:31:47.636691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:47.636691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:31:47.636691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:31:47.636691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:31:47.636691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:31:47.636691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:31:47.636691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:47.636691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 19:31:47.636691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:47.636691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:47.636691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:47.636691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:47.636691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:47.636691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:47.636691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:47.636691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:47.637950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:47.637950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:47.637950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:47.637950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:47.637950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:47.637950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:47.637950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:31:47.637950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:31:47.637950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:31:47.637950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:31:47.637950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:47.637950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:47.637950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:47.637950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:47.637950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:47.637950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:47.638816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:47.638816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:47.638816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:47.638816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:47.638816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:47.638816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:47.638816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:47.638816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:47.638816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:47.638816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:47.638816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:31:47.638816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:47.638816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 19:31:47.638816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:47.638816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 500/999] 10.222.0.13 () {56 vars in 1126 bytes} [Fri May 27 19:31:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:47.646807 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb74602-8d47-4d44-b8d0-e9f85d71f496 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:47.761973 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb74602-8d47-4d44-b8d0-e9f85d71f496 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:47.800498 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d30a99-fdc6-4f5f-93f7-369e4b3a8b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:47.801206 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d30a99-fdc6-4f5f-93f7-369e4b3a8b75 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:47.801489 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d30a99-fdc6-4f5f-93f7-369e4b3a8b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:47.801765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d30a99-fdc6-4f5f-93f7-369e4b3a8b75 None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:47.802526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d30a99-fdc6-4f5f-93f7-369e4b3a8b75 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:47.802790 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d30a99-fdc6-4f5f-93f7-369e4b3a8b75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:47.804224 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d30a99-fdc6-4f5f-93f7-369e4b3a8b75 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:47.814789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 500/1000] 10.222.0.13 () {56 vars in 1122 bytes} [Fri May 27 19:31:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:47.816463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5974d-432f-4366-9a20-fc62e1eb0c7b None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:47.924413 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5974d-432f-4366-9a20-fc62e1eb0c7b None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:47.931866 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:47.932616 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df43fd4-abb1-4958-b87a-a96903583f0e None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:47.933486 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df43fd4-abb1-4958-b87a-a96903583f0e None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:47.933761 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df43fd4-abb1-4958-b87a-a96903583f0e None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:47.935482 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 501/1001] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:47.940576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58c897-1c61-4da3-8d15-7ef30e9a6b61 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:47.945430 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58c897-1c61-4da3-8d15-7ef30e9a6b61 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:47.945430 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c58c897-1c61-4da3-8d15-7ef30e9a6b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:47.958711 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-5c58c897-1c61-4da3-8d15-7ef30e9a6b61 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:47.959924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-5c58c897-1c61-4da3-8d15-7ef30e9a6b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:47.968463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb2179-d29e-4dcd-9ba4-59b6c6b862ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:47.969426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb2179-d29e-4dcd-9ba4-59b6c6b862ce None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:47.969704 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb2179-d29e-4dcd-9ba4-59b6c6b862ce None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:47.969974 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb2179-d29e-4dcd-9ba4-59b6c6b862ce None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:47.970902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb2179-d29e-4dcd-9ba4-59b6c6b862ce None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:47.971209 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb2179-d29e-4dcd-9ba4-59b6c6b862ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:47.972705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb2179-d29e-4dcd-9ba4-59b6c6b862ce None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:47.983074 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 501/1002] 10.222.0.13 () {56 vars in 1122 bytes} [Fri May 27 19:31:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:47.984253 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb2179-d29e-4dcd-9ba4-59b6c6b862ce None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:48.024528 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:48.044581 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 502/1003] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 19:31:48.067715 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58c897-1c61-4da3-8d15-7ef30e9a6b61 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:48.110763 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb2179-d29e-4dcd-9ba4-59b6c6b862ce None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:48.151890 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446ab73-6534-459b-8389-72205c533d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:48.152650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446ab73-6534-459b-8389-72205c533d0e None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:48.152936 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446ab73-6534-459b-8389-72205c533d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:48.153243 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446ab73-6534-459b-8389-72205c533d0e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:48.154097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446ab73-6534-459b-8389-72205c533d0e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:48.155524 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446ab73-6534-459b-8389-72205c533d0e None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:48.172553 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c58c897-1c61-4da3-8d15-7ef30e9a6b61 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:48.214671 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33678808-8b9d-4d8f-bc5d-fcb3c72760fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:48.215584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33678808-8b9d-4d8f-bc5d-fcb3c72760fe None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:48.215869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33678808-8b9d-4d8f-bc5d-fcb3c72760fe None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:48.216143 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33678808-8b9d-4d8f-bc5d-fcb3c72760fe None admin] PATH_INFO: `/v3/services/image` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:48.216802 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33678808-8b9d-4d8f-bc5d-fcb3c72760fe None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:48.218212 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33678808-8b9d-4d8f-bc5d-fcb3c72760fe None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:48.230882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-33678808-8b9d-4d8f-bc5d-fcb3c72760fe None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 19:31:48.230882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:48.230882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:31:48.230882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:31:48.230882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:31:48.230882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:31:48.230882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:31:48.230882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:48.230882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 19:31:48.230882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:48.230882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:48.230882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:48.230882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:48.230882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:48.230882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:48.230882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:48.230882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:48.231963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:48.231963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:48.231963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:48.231963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:48.231963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:48.231963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:48.231963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:31:48.231963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:31:48.231963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:31:48.231963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:31:48.231963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:48.231963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:48.231963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:48.231963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:48.231963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:48.231963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:48.232979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:48.232979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:48.232979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:48.232979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:48.232979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:48.232979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:48.232979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:48.232979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:48.232979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:48.232979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:48.232979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:31:48.232979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:48.232979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 19:31:48.232979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:48.232979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 503/1004] 10.222.0.13 () {56 vars in 1105 bytes} [Fri May 27 19:31:48 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:31:48.238230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33678808-8b9d-4d8f-bc5d-fcb3c72760fe None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:48.292975 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 502/1005] 10.222.0.13 () {60 vars in 1144 bytes} [Fri May 27 19:31:47 2022] POST /identity/v3/endpoints => generated 325 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:31:48.295020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:48.359981 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33678808-8b9d-4d8f-bc5d-fcb3c72760fe None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:48.414563 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9008ee-050e-4f2c-9d48-6411779294b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:48.415716 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9008ee-050e-4f2c-9d48-6411779294b8 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:48.415716 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9008ee-050e-4f2c-9d48-6411779294b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:48.415716 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9008ee-050e-4f2c-9d48-6411779294b8 None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:48.416461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9008ee-050e-4f2c-9d48-6411779294b8 None admin] RBAC: Adding query filter params (name=image) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:48.418472 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9008ee-050e-4f2c-9d48-6411779294b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:48.419401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9008ee-050e-4f2c-9d48-6411779294b8 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:48.426137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 504/1006] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:31:48.432416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446ab73-6534-459b-8389-72205c533d0e None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:48.471955 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:48.573941 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446ab73-6534-459b-8389-72205c533d0e None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:48.611786 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a3072-e584-4541-b9cb-e1e34a93db1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:48.612460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a3072-e584-4541-b9cb-e1e34a93db1b None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:48.612747 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a3072-e584-4541-b9cb-e1e34a93db1b None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:48.613051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a3072-e584-4541-b9cb-e1e34a93db1b None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:48.613764 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a3072-e584-4541-b9cb-e1e34a93db1b None admin] RBAC: Adding query filter params (type=image) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:48.614070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a3072-e584-4541-b9cb-e1e34a93db1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:48.615444 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a3072-e584-4541-b9cb-e1e34a93db1b None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:48.625046 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 503/1007] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:48 2022] GET /identity/v3/services?type=image => generated 122 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:31:49.632636 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f321559-70d7-4718-8777-296a7caa1703 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:49.633287 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f321559-70d7-4718-8777-296a7caa1703 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:49.633287 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f321559-70d7-4718-8777-296a7caa1703 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:49.634620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 505/1008] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:49.640903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be34eee-7db7-42fc-b3bb-d15fbaacd1ff None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:49.641430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be34eee-7db7-42fc-b3bb-d15fbaacd1ff None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:49.641709 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be34eee-7db7-42fc-b3bb-d15fbaacd1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:49.659773 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-3be34eee-7db7-42fc-b3bb-d15fbaacd1ff None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:49.661021 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-3be34eee-7db7-42fc-b3bb-d15fbaacd1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:49.740080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 504/1009] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:49 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:49.763718 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f321559-70d7-4718-8777-296a7caa1703 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:49.877990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f321559-70d7-4718-8777-296a7caa1703 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:49.889750 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:49.914516 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb5bffc-3f77-4ba6-8ec1-480904ee7254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:49.915369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5bffc-3f77-4ba6-8ec1-480904ee7254 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:49.915647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5bffc-3f77-4ba6-8ec1-480904ee7254 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:49.915921 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5bffc-3f77-4ba6-8ec1-480904ee7254 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:49.916576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb5bffc-3f77-4ba6-8ec1-480904ee7254 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:49.918013 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb5bffc-3f77-4ba6-8ec1-480904ee7254 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:49.930236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-cfb5bffc-3f77-4ba6-8ec1-480904ee7254 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 19:31:49.930236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:49.930236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:31:49.930236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:31:49.930236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:31:49.930236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:31:49.930236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:31:49.930236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:49.930236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 19:31:49.930236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:49.930236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:49.930236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:49.930236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:49.930236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:49.930236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:49.930236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:49.930236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:49.931227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:49.931227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:49.931227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:49.931227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:49.931227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:49.931227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:49.931227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:31:49.931227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:31:49.931227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:31:49.931227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:31:49.931227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:49.931227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:49.931227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:49.931227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:49.931227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:49.931227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:49.938177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:49.938177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:49.938177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:49.938177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:49.938177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:49.938177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:49.938177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:49.938177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:49.938177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:49.938177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:49.938177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:31:49.938177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:49.938177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 19:31:49.938177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:49.938177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 506/1010] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:49.941917 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be34eee-7db7-42fc-b3bb-d15fbaacd1ff None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:49.967933 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:50.044326 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be34eee-7db7-42fc-b3bb-d15fbaacd1ff None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:50.083701 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68f2c5-46c7-453b-b4f4-9da4bc89bf43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:50.084369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68f2c5-46c7-453b-b4f4-9da4bc89bf43 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:50.084655 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68f2c5-46c7-453b-b4f4-9da4bc89bf43 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:50.084939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68f2c5-46c7-453b-b4f4-9da4bc89bf43 None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:50.085680 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd68f2c5-46c7-453b-b4f4-9da4bc89bf43 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:50.085982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd68f2c5-46c7-453b-b4f4-9da4bc89bf43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:50.087366 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd68f2c5-46c7-453b-b4f4-9da4bc89bf43 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:50.097038 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 505/1011] 10.222.0.13 () {56 vars in 1112 bytes} [Fri May 27 19:31:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:50.099680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb5bffc-3f77-4ba6-8ec1-480904ee7254 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:50.209896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb5bffc-3f77-4ba6-8ec1-480904ee7254 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:50.252068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e0c1ad-594a-4718-9009-e0e08f41f618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:50.252746 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0c1ad-594a-4718-9009-e0e08f41f618 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:50.253058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0c1ad-594a-4718-9009-e0e08f41f618 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:50.253338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e0c1ad-594a-4718-9009-e0e08f41f618 None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:50.254081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0c1ad-594a-4718-9009-e0e08f41f618 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:50.254389 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0c1ad-594a-4718-9009-e0e08f41f618 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:50.255758 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e0c1ad-594a-4718-9009-e0e08f41f618 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:50.266099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 507/1012] 10.222.0.13 () {56 vars in 1112 bytes} [Fri May 27 19:31:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:31:50.267319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68f2c5-46c7-453b-b4f4-9da4bc89bf43 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:50.358707 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:50.370877 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68f2c5-46c7-453b-b4f4-9da4bc89bf43 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:50.405539 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c86f3-68ba-4c24-bdd6-2aa6bcd0fe71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:50.406488 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c86f3-68ba-4c24-bdd6-2aa6bcd0fe71 None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:50.406807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c86f3-68ba-4c24-bdd6-2aa6bcd0fe71 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:50.407121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c86f3-68ba-4c24-bdd6-2aa6bcd0fe71 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:50.408013 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3c86f3-68ba-4c24-bdd6-2aa6bcd0fe71 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:50.409877 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3c86f3-68ba-4c24-bdd6-2aa6bcd0fe71 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:50.471883 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 506/1013] 10.222.0.13 () {60 vars in 1144 bytes} [Fri May 27 19:31:50 2022] POST /identity/v3/endpoints => generated 334 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:31:50.646275 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467f911-2488-48fe-8015-a6e81e61a82a None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:50.646781 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467f911-2488-48fe-8015-a6e81e61a82a None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:50.646781 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467f911-2488-48fe-8015-a6e81e61a82a None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:50.648141 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 508/1014] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:50.654772 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a741c08-5e55-45df-b4ef-978460c0ef36 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:50.655246 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a741c08-5e55-45df-b4ef-978460c0ef36 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:50.655438 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a741c08-5e55-45df-b4ef-978460c0ef36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:50.673442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-6a741c08-5e55-45df-b4ef-978460c0ef36 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:50.674636 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-6a741c08-5e55-45df-b4ef-978460c0ef36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:50.759948 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 507/1015] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:50 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:50.785170 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467f911-2488-48fe-8015-a6e81e61a82a None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:50.831713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:50.891328 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91053eb9-b315-43d9-ae15-64591868a2f4 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:50.891617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91053eb9-b315-43d9-ae15-64591868a2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:50.891908 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91053eb9-b315-43d9-ae15-64591868a2f4 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:50.893347 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 508/1016] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 19:31:50.896561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467f911-2488-48fe-8015-a6e81e61a82a None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:50.898463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc1e54-9a8c-46c7-8ad2-4f9a2290b0fc None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:50.898758 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc1e54-9a8c-46c7-8ad2-4f9a2290b0fc None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:50.899044 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc1e54-9a8c-46c7-8ad2-4f9a2290b0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:50.916560 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-76cc1e54-9a8c-46c7-8ad2-4f9a2290b0fc None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:50.917783 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-76cc1e54-9a8c-46c7-8ad2-4f9a2290b0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:50.936526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b8c27-50ce-499b-aa5b-ea40ac12039b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:50.937533 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b8c27-50ce-499b-aa5b-ea40ac12039b None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:50.937821 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b8c27-50ce-499b-aa5b-ea40ac12039b None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:50.938416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b8c27-50ce-499b-aa5b-ea40ac12039b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:50.939616 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b8c27-50ce-499b-aa5b-ea40ac12039b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:50.941526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b8c27-50ce-499b-aa5b-ea40ac12039b None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:50.943472 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 509/1017] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:50.950155 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b8c27-50ce-499b-aa5b-ea40ac12039b None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:50.988406 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:50.993761 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 509/1018] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:50 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 19:31:51.017090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc1e54-9a8c-46c7-8ad2-4f9a2290b0fc None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:51.063611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b8c27-50ce-499b-aa5b-ea40ac12039b None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:51.071950 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:51.105551 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b62ea-a423-4b5c-bad6-cac259b13058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:51.106246 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b62ea-a423-4b5c-bad6-cac259b13058 None admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:51.106538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b62ea-a423-4b5c-bad6-cac259b13058 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:51.106817 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b62ea-a423-4b5c-bad6-cac259b13058 None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:51.107542 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b62ea-a423-4b5c-bad6-cac259b13058 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:51.109149 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b62ea-a423-4b5c-bad6-cac259b13058 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:51.136414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc1e54-9a8c-46c7-8ad2-4f9a2290b0fc None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:51.173806 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db2486-f63b-4729-97c8-20bd65104184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:51.174388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db2486-f63b-4729-97c8-20bd65104184 None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:51.174684 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db2486-f63b-4729-97c8-20bd65104184 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:51.174958 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db2486-f63b-4729-97c8-20bd65104184 None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:51.175694 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db2486-f63b-4729-97c8-20bd65104184 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:51.177186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db2486-f63b-4729-97c8-20bd65104184 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:51.222790 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 510/1019] 10.222.0.13 () {60 vars in 1141 bytes} [Fri May 27 19:31:50 2022] POST /identity/v3/projects => generated 311 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:31:51.304989 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 510/1020] 10.222.0.13 () {60 vars in 1141 bytes} [Fri May 27 19:31:51 2022] POST /identity/v3/services => generated 242 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:31:52.840367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4df2ea-d21f-44cd-b5b7-58ef477ab437 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:52.840914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4df2ea-d21f-44cd-b5b7-58ef477ab437 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:52.840914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4df2ea-d21f-44cd-b5b7-58ef477ab437 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:52.844999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 511/1021] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:52.847623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4923d7a-5568-4dec-b870-2c433cf55dd7 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:52.848051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4923d7a-5568-4dec-b870-2c433cf55dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:52.848234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4923d7a-5568-4dec-b870-2c433cf55dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:52.869104 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-b4923d7a-5568-4dec-b870-2c433cf55dd7 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:52.870760 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-b4923d7a-5568-4dec-b870-2c433cf55dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:52.934906 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:52.953307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 511/1022] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:52 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:52.977667 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4df2ea-d21f-44cd-b5b7-58ef477ab437 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:53.096906 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4df2ea-d21f-44cd-b5b7-58ef477ab437 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:53.135283 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989fde8-93e9-46eb-8486-52e1f8cf110d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:53.136533 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989fde8-93e9-46eb-8486-52e1f8cf110d None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:53.138141 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989fde8-93e9-46eb-8486-52e1f8cf110d None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:53.138141 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4989fde8-93e9-46eb-8486-52e1f8cf110d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:53.148842 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4989fde8-93e9-46eb-8486-52e1f8cf110d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:31:53.157576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-4989fde8-93e9-46eb-8486-52e1f8cf110d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 19:31:53.157576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:53.157576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:53.157576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:53.157576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:53.157576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:53.157576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:53.157576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:53.157576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:53.157576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:53.157576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:53.157576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:53.157576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:31:53.157576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:31:53.157576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:31:53.157576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:31:53.158546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:31:53.158546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:31:53.158546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:31:53.158546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:31:53.158546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:53.158546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:53.158546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:53.158546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:53.158546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:53.158546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:53.158546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:53.158546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:53.158546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:53.158546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:53.158546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:53.158546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:53.159494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:53.159494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:53.159494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:53.159494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:53.159494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:31:53.159494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:31:53.159494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:31:53.159494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:31:53.159494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:31:53.159494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:31:53.159494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 19:31:53.159494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:53.161315 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 512/1023] 10.222.0.13 () {56 vars in 1120 bytes} [Fri May 27 19:31:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:53.164807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4923d7a-5568-4dec-b870-2c433cf55dd7 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:53.276374 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4923d7a-5568-4dec-b870-2c433cf55dd7 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:53.318243 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6779e5b-bafe-4ca1-b811-55cfe4a7cdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:53.318750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6779e5b-bafe-4ca1-b811-55cfe4a7cdc0 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:53.319131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6779e5b-bafe-4ca1-b811-55cfe4a7cdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:53.319322 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6779e5b-bafe-4ca1-b811-55cfe4a7cdc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:53.320052 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6779e5b-bafe-4ca1-b811-55cfe4a7cdc0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:53.320357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6779e5b-bafe-4ca1-b811-55cfe4a7cdc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:53.322761 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6779e5b-bafe-4ca1-b811-55cfe4a7cdc0 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:53.336993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 512/1024] 10.222.0.13 () {56 vars in 1115 bytes} [Fri May 27 19:31:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:53.340744 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989fde8-93e9-46eb-8486-52e1f8cf110d None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:53.363753 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:53.454775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4989fde8-93e9-46eb-8486-52e1f8cf110d None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:53.491880 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5c287-cb1e-4b0b-82cf-0964af2a880e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:53.492723 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5c287-cb1e-4b0b-82cf-0964af2a880e None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:53.493029 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5c287-cb1e-4b0b-82cf-0964af2a880e None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:53.493307 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5c287-cb1e-4b0b-82cf-0964af2a880e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:53.504794 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5c287-cb1e-4b0b-82cf-0964af2a880e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:53.508416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5c287-cb1e-4b0b-82cf-0964af2a880e None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:53.525917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-9cc5c287-cb1e-4b0b-82cf-0964af2a880e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:53.525917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:53.525917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:53.525917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:53.525917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:53.525917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:53.525917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:53.525917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:53.525917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:53.525917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:53.525917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:53.525917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:53.525917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:53.525917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:53.525917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:53.532100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:53.532100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:53.532100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:53.532100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:53.532100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:53.532100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:53.532100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:53.532100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:53.532100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:53.532100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:53.532100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:53.532100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:53.532100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:53.532100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:53.532100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:53.532100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:53.532100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:53.533404 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:53.533404 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:53.533404 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:53.533404 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:53.533404 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 513/1025] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:53.533752 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef3fe8-edc0-4e3d-8678-1114276f9462 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:53.533752 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef3fe8-edc0-4e3d-8678-1114276f9462 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:53.533752 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef3fe8-edc0-4e3d-8678-1114276f9462 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:53.534975 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 513/1026] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:53.535263 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5c287-cb1e-4b0b-82cf-0964af2a880e None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:53.540360 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e90c45-66b6-440b-8255-37e50dae2fc3 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:53.540652 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e90c45-66b6-440b-8255-37e50dae2fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:53.541000 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e90c45-66b6-440b-8255-37e50dae2fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:53.559868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-b9e90c45-66b6-440b-8255-37e50dae2fc3 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:53.561152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-b9e90c45-66b6-440b-8255-37e50dae2fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:53.611837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:53.626486 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:53.638131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 514/1027] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 19:31:53.639646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822eedb-d3ad-4a0f-954b-59d851eff803 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:53.639942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822eedb-d3ad-4a0f-954b-59d851eff803 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:53.640212 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822eedb-d3ad-4a0f-954b-59d851eff803 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:53.645435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 515/1028] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:53.647590 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc93d4-b199-4c96-853c-aa4a345b6485 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:53.647944 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc93d4-b199-4c96-853c-aa4a345b6485 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:53.648310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc93d4-b199-4c96-853c-aa4a345b6485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:53.659123 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5c287-cb1e-4b0b-82cf-0964af2a880e None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:53.665826 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-44fc93d4-b199-4c96-853c-aa4a345b6485 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:53.667688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-44fc93d4-b199-4c96-853c-aa4a345b6485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:53.699135 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef84a3c-ea66-40fb-9356-95fbf578eae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:53.699810 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef84a3c-ea66-40fb-9356-95fbf578eae7 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:53.700092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef84a3c-ea66-40fb-9356-95fbf578eae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:53.700648 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef84a3c-ea66-40fb-9356-95fbf578eae7 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:53.701430 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef84a3c-ea66-40fb-9356-95fbf578eae7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:53.701737 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef84a3c-ea66-40fb-9356-95fbf578eae7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:53.703150 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef84a3c-ea66-40fb-9356-95fbf578eae7 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:53.716039 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 514/1029] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:53.718155 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef84a3c-ea66-40fb-9356-95fbf578eae7 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:53.748683 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 516/1030] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 19:31:53.749584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc93d4-b199-4c96-853c-aa4a345b6485 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:53.840072 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef84a3c-ea66-40fb-9356-95fbf578eae7 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:53.856059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fc93d4-b199-4c96-853c-aa4a345b6485 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:53.866390 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:53.870323 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:53.876247 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eaf347-a944-4948-9da6-b1218331fd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:53.877143 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eaf347-a944-4948-9da6-b1218331fd0a None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:53.877382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eaf347-a944-4948-9da6-b1218331fd0a None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:53.877661 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eaf347-a944-4948-9da6-b1218331fd0a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:53.878847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eaf347-a944-4948-9da6-b1218331fd0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:53.880327 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eaf347-a944-4948-9da6-b1218331fd0a None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:53.882036 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 515/1031] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:53.882809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eaf347-a944-4948-9da6-b1218331fd0a None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:53.893556 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a1d34-bd66-4d5c-a02b-493573d77dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:53.894355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a1d34-bd66-4d5c-a02b-493573d77dec None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:53.894632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a1d34-bd66-4d5c-a02b-493573d77dec None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:53.894908 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a1d34-bd66-4d5c-a02b-493573d77dec None admin] PATH_INFO: `/v3/users/nova` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:53.903503 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a1d34-bd66-4d5c-a02b-493573d77dec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:53.910194 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a1d34-bd66-4d5c-a02b-493573d77dec None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:53.921118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-ee9a1d34-bd66-4d5c-a02b-493573d77dec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 19:31:53.921118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:53.921118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:53.921118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:53.921118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:53.921118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:53.921118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:53.921118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:53.921118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:53.921118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:53.921118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:53.921118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:53.921118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:31:53.921118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:31:53.921118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:31:53.921118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:31:53.922735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:53.922735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:53.922735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:31:53.922735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:53.922735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:31:53.922735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:53.922735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:53.922735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:53.922735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:53.922735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:53.922735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:53.922735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:53.922735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:53.922735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:53.922735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:53.922735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:53.922735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:53.922735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:53.923865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:53.923865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:53.923865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:31:53.923865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:31:53.923865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:31:53.923865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:31:53.923865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:31:53.923865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:31:53.923865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 19:31:53.923865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:53.923865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 517/1032] 10.222.0.13 () {56 vars in 1093 bytes} [Fri May 27 19:31:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:53.923865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a1d34-bd66-4d5c-a02b-493573d77dec None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:53.992610 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eaf347-a944-4948-9da6-b1218331fd0a None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:54.030587 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a1d34-bd66-4d5c-a02b-493573d77dec None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:54.030808 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff44a0-b4d0-48f7-bae2-1dad26a67117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:54.031457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff44a0-b4d0-48f7-bae2-1dad26a67117 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:54.031744 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff44a0-b4d0-48f7-bae2-1dad26a67117 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:54.032049 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff44a0-b4d0-48f7-bae2-1dad26a67117 None admin] PATH_INFO: `/v3/services/image` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:54.032711 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ff44a0-b4d0-48f7-bae2-1dad26a67117 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:54.034263 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ff44a0-b4d0-48f7-bae2-1dad26a67117 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:54.047018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-c9ff44a0-b4d0-48f7-bae2-1dad26a67117 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 19:31:54.047018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:54.047018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:31:54.047018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:31:54.047018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:31:54.047018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:31:54.047018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:31:54.047018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:54.047018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 19:31:54.047018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:54.047018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:54.047018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:54.047018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:54.047018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:54.047018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:54.047018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:54.047018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:54.048176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:54.048176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:54.048176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:54.048176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:54.048176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:54.048176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:54.048176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:31:54.048176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:31:54.048176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:31:54.048176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:31:54.048176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:54.048176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:54.048176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:54.048176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:54.048176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:54.048176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:54.049920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:54.049920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:54.049920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:54.049920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:54.049920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:54.049920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:54.049920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:54.049920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:54.049920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:54.049920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:54.049920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:31:54.049920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:54.049920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 19:31:54.049920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:54.049920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 516/1033] 10.222.0.13 () {56 vars in 1105 bytes} [Fri May 27 19:31:53 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:54.051003 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff44a0-b4d0-48f7-bae2-1dad26a67117 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:54.071448 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f29c2d-13ee-4ad0-941e-2e34aca9e8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:54.072169 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f29c2d-13ee-4ad0-941e-2e34aca9e8f3 None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:54.072583 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f29c2d-13ee-4ad0-941e-2e34aca9e8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:54.073124 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f29c2d-13ee-4ad0-941e-2e34aca9e8f3 None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:54.073837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f29c2d-13ee-4ad0-941e-2e34aca9e8f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:54.075557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f29c2d-13ee-4ad0-941e-2e34aca9e8f3 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:54.116353 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:54.155770 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff44a0-b4d0-48f7-bae2-1dad26a67117 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:54.189465 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251b6ae-e6c0-4891-957f-a694490e42a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:54.190144 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251b6ae-e6c0-4891-957f-a694490e42a2 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:54.190430 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251b6ae-e6c0-4891-957f-a694490e42a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:54.190710 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251b6ae-e6c0-4891-957f-a694490e42a2 None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:54.191544 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251b6ae-e6c0-4891-957f-a694490e42a2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:54.191860 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251b6ae-e6c0-4891-957f-a694490e42a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:54.193395 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251b6ae-e6c0-4891-957f-a694490e42a2 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:54.194096 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 518/1034] 10.222.0.13 () {60 vars in 1132 bytes} [Fri May 27 19:31:53 2022] POST /identity/v3/users => generated 285 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:31:54.195129 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f29c2d-13ee-4ad0-941e-2e34aca9e8f3 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:54.215740 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 517/1035] 10.222.0.13 () {56 vars in 1133 bytes} [Fri May 27 19:31:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:54.222086 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251b6ae-e6c0-4891-957f-a694490e42a2 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:54.246796 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:54.345345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f29c2d-13ee-4ad0-941e-2e34aca9e8f3 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:54.360315 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251b6ae-e6c0-4891-957f-a694490e42a2 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:54.386256 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:54.388919 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998f8dc-d4a0-4d85-bd63-ef909419a56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:54.389891 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998f8dc-d4a0-4d85-bd63-ef909419a56a None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:54.390181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998f8dc-d4a0-4d85-bd63-ef909419a56a None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:54.390462 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998f8dc-d4a0-4d85-bd63-ef909419a56a None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:54.391232 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998f8dc-d4a0-4d85-bd63-ef909419a56a None admin] RBAC: Adding query filter params (name=image) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:54.391595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998f8dc-d4a0-4d85-bd63-ef909419a56a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:54.393017 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998f8dc-d4a0-4d85-bd63-ef909419a56a None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:54.399177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 519/1036] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:54.405759 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88132b-64bf-4a1f-b01d-5ec2d708fa7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:54.406507 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88132b-64bf-4a1f-b01d-5ec2d708fa7a None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:54.406722 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88132b-64bf-4a1f-b01d-5ec2d708fa7a None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:54.406997 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88132b-64bf-4a1f-b01d-5ec2d708fa7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:54.407196 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998f8dc-d4a0-4d85-bd63-ef909419a56a None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:54.496535 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88132b-64bf-4a1f-b01d-5ec2d708fa7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:54.498039 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88132b-64bf-4a1f-b01d-5ec2d708fa7a None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:54.522409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:54.537307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998f8dc-d4a0-4d85-bd63-ef909419a56a None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:54.575734 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc5390-546e-41f6-be5c-5b9fb36db607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:54.576407 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc5390-546e-41f6-be5c-5b9fb36db607 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:54.576688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc5390-546e-41f6-be5c-5b9fb36db607 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:54.576984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc5390-546e-41f6-be5c-5b9fb36db607 None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:54.577769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc5390-546e-41f6-be5c-5b9fb36db607 None admin] RBAC: Adding query filter params (type=image) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:54.578075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc5390-546e-41f6-be5c-5b9fb36db607 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:54.579458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc5390-546e-41f6-be5c-5b9fb36db607 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:54.590790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 520/1037] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:31:54.592055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc5390-546e-41f6-be5c-5b9fb36db607 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:54.611210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 518/1038] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:31:54 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 19:31:54.617268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88132b-64bf-4a1f-b01d-5ec2d708fa7a None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:54.641145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:54.699842 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc5390-546e-41f6-be5c-5b9fb36db607 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:54.727063 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88132b-64bf-4a1f-b01d-5ec2d708fa7a None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:54.738283 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335535a-269e-49f8-be82-ef17ada61a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:54.739264 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335535a-269e-49f8-be82-ef17ada61a5f None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:54.739691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335535a-269e-49f8-be82-ef17ada61a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:54.740106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335535a-269e-49f8-be82-ef17ada61a5f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:54.740904 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335535a-269e-49f8-be82-ef17ada61a5f None admin] RBAC: Adding query filter params (interface=public, service_id=217fa89c88544888be36ab7bf42b8a16, region_id=RegionOne) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:54.741408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335535a-269e-49f8-be82-ef17ada61a5f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:54.743489 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335535a-269e-49f8-be82-ef17ada61a5f None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:54.749932 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 521/1039] 10.222.0.13 () {56 vars in 1251 bytes} [Fri May 27 19:31:54 2022] GET /identity/v3/endpoints?service_id=217fa89c88544888be36ab7bf42b8a16&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:31:54.755509 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335535a-269e-49f8-be82-ef17ada61a5f None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:54.765501 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68b738-1473-4446-9756-d8fa89aa37aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:54.766898 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68b738-1473-4446-9756-d8fa89aa37aa None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:54.767405 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68b738-1473-4446-9756-d8fa89aa37aa None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:54.767884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68b738-1473-4446-9756-d8fa89aa37aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:54.774668 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:54.784934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68b738-1473-4446-9756-d8fa89aa37aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:54.787846 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68b738-1473-4446-9756-d8fa89aa37aa None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:54.809658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-8f68b738-1473-4446-9756-d8fa89aa37aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:54.809658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:54.809658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:54.809658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:54.809658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:54.809658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:54.809658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:54.809658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:54.809658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:54.809658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:54.809658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:54.809658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:54.809658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:54.809658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:54.809658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:54.810696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:54.810696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:54.810696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:54.810696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:54.810696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:54.810696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:54.810696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:54.810696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:54.810696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:54.810696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:54.810696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:54.810696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:54.810696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:54.810696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:54.810696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:54.810696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:54.810696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:54.811693 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:54.811693 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:54.811693 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:54.811693 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:54.811693 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 519/1040] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:31:54.817047 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68b738-1473-4446-9756-d8fa89aa37aa None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:54.869887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335535a-269e-49f8-be82-ef17ada61a5f None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:54.890772 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:54.907495 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514173b-3b5c-470e-bd61-9561c2a4830b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:54.908167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514173b-3b5c-470e-bd61-9561c2a4830b None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:54.908455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514173b-3b5c-470e-bd61-9561c2a4830b None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:54.908735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6514173b-3b5c-470e-bd61-9561c2a4830b None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:54.911325 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514173b-3b5c-470e-bd61-9561c2a4830b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:54.911325 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6514173b-3b5c-470e-bd61-9561c2a4830b None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:54.918371 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 522/1041] 10.222.0.13 () {56 vars in 1087 bytes} [Fri May 27 19:31:54 2022] GET /identity/v3/services => generated 1957 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:31:54.923085 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68b738-1473-4446-9756-d8fa89aa37aa None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:54.964326 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7719b-e64d-44d2-8847-9f084601e891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:54.964915 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7719b-e64d-44d2-8847-9f084601e891 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:54.965229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7719b-e64d-44d2-8847-9f084601e891 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:54.965509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7719b-e64d-44d2-8847-9f084601e891 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:54.966250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7719b-e64d-44d2-8847-9f084601e891 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:54.966559 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7719b-e64d-44d2-8847-9f084601e891 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:54.967980 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7719b-e64d-44d2-8847-9f084601e891 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:54.981223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 520/1042] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:54.986900 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514173b-3b5c-470e-bd61-9561c2a4830b None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:55.079963 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:55.101885 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6514173b-3b5c-470e-bd61-9561c2a4830b None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:55.140518 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92682d7-cdc2-4192-89c7-8abeaf528d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:55.141351 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92682d7-cdc2-4192-89c7-8abeaf528d0e None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:55.141714 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92682d7-cdc2-4192-89c7-8abeaf528d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:55.142003 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92682d7-cdc2-4192-89c7-8abeaf528d0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:55.148832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92682d7-cdc2-4192-89c7-8abeaf528d0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:55.151027 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92682d7-cdc2-4192-89c7-8abeaf528d0e None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:55.165060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-c92682d7-cdc2-4192-89c7-8abeaf528d0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:55.165060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:55.165060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:55.165060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:55.165060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:55.165060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:55.165060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:55.165060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:55.165060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:55.165060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:55.165060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:55.165060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:55.165060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:31:55.165060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:31:55.165060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:31:55.166112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:31:55.166112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:55.166112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:55.166112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:55.166112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:55.166112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:55.166112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:55.166112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:55.166112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:55.166112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:55.166112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:55.166112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:55.166112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:55.166112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:55.166112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:55.166112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:55.166112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:55.167044 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:31:55.167044 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:31:55.167044 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:31:55.167044 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:31:55.167044 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:31:55.167044 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:31:55.167044 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:31:55.167044 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:55.167044 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 523/1043] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:55.173531 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7719b-e64d-44d2-8847-9f084601e891 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:55.283003 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7719b-e64d-44d2-8847-9f084601e891 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:55.320543 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:55.327825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991dfc4-8a4e-45ec-9f68-758e990f4d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:55.328505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991dfc4-8a4e-45ec-9f68-758e990f4d1f None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:55.328791 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991dfc4-8a4e-45ec-9f68-758e990f4d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:55.329103 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991dfc4-8a4e-45ec-9f68-758e990f4d1f None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:55.329903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d991dfc4-8a4e-45ec-9f68-758e990f4d1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:55.330216 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d991dfc4-8a4e-45ec-9f68-758e990f4d1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:55.331659 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d991dfc4-8a4e-45ec-9f68-758e990f4d1f None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:55.345499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 521/1044] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:31:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:55.351242 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92682d7-cdc2-4192-89c7-8abeaf528d0e None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:55.467431 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92682d7-cdc2-4192-89c7-8abeaf528d0e None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:55.501798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203e65d-c939-4915-962f-8548a937e898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:55.502457 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203e65d-c939-4915-962f-8548a937e898 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:55.502739 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203e65d-c939-4915-962f-8548a937e898 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:55.503013 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203e65d-c939-4915-962f-8548a937e898 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:55.503844 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203e65d-c939-4915-962f-8548a937e898 None admin] RBAC: Adding query filter params (role.id=30c027fab0344b47b1ba1ef6c6cbd818, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=7542e1fd160e4b2c9d25accf19aad19f) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:55.504233 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203e65d-c939-4915-962f-8548a937e898 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:55.508331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203e65d-c939-4915-962f-8548a937e898 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:55.515071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 524/1045] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:31:55 2022] GET /identity/v3/role_assignments?user.id=7542e1fd160e4b2c9d25accf19aad19f&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=30c027fab0344b47b1ba1ef6c6cbd818 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:56.571045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766687d-619d-4ec0-94b9-1a6449d8301a None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:56.571604 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766687d-619d-4ec0-94b9-1a6449d8301a None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:56.571727 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4766687d-619d-4ec0-94b9-1a6449d8301a None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:56.573442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 522/1046] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:56.579154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca92055-7f67-4d5f-a0a4-18b04b4a1f50 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:56.579607 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca92055-7f67-4d5f-a0a4-18b04b4a1f50 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:56.579784 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca92055-7f67-4d5f-a0a4-18b04b4a1f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:56.585727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:56.605392 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-8ca92055-7f67-4d5f-a0a4-18b04b4a1f50 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:56.607482 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-8ca92055-7f67-4d5f-a0a4-18b04b4a1f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:56.695906 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 525/1047] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:56.720391 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766687d-619d-4ec0-94b9-1a6449d8301a None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:56.885052 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4766687d-619d-4ec0-94b9-1a6449d8301a None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:56.925244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3d1a9-fc3c-4859-aae8-f8df73f20d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:56.926043 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3d1a9-fc3c-4859-aae8-f8df73f20d4a None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:56.926364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3d1a9-fc3c-4859-aae8-f8df73f20d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:56.926596 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3d1a9-fc3c-4859-aae8-f8df73f20d4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:56.942970 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3d1a9-fc3c-4859-aae8-f8df73f20d4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:31:56.951775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-30f3d1a9-fc3c-4859-aae8-f8df73f20d4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:31:56.951775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:56.951775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:56.951775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:56.951775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:56.951775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:56.951775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:56.951775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:56.951775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:56.951775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:56.951775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:56.951775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:56.951775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:31:56.951775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:31:56.951775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:31:56.951775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:31:56.952877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:31:56.952877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:31:56.952877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:31:56.952877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:31:56.952877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:56.952877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:56.952877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:56.952877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:56.952877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:56.952877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:56.952877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:56.952877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:31:56.952877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:56.952877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:56.952877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:56.952877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:56.959552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:56.959552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:56.959552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:56.959552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:56.959552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:31:56.959552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:31:56.959552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:31:56.959552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:31:56.959552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:31:56.959552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:31:56.959552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:31:56.959552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:31:56.959552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 523/1048] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:31:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:56.960643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca92055-7f67-4d5f-a0a4-18b04b4a1f50 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:57.087202 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca92055-7f67-4d5f-a0a4-18b04b4a1f50 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:57.125864 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ea8993-b29c-4278-9bb3-a272d80082fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:57.126587 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea8993-b29c-4278-9bb3-a272d80082fe None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:57.126882 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea8993-b29c-4278-9bb3-a272d80082fe None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:57.127162 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ea8993-b29c-4278-9bb3-a272d80082fe None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:57.127917 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ea8993-b29c-4278-9bb3-a272d80082fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:57.128231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ea8993-b29c-4278-9bb3-a272d80082fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:57.129688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ea8993-b29c-4278-9bb3-a272d80082fe None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:57.131262 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:57.146251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 526/1049] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:31:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:57.146561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3d1a9-fc3c-4859-aae8-f8df73f20d4a None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:57.205774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba392430-43f6-4b39-aa29-0ae407b2b156 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:57.206096 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba392430-43f6-4b39-aa29-0ae407b2b156 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:57.206346 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba392430-43f6-4b39-aa29-0ae407b2b156 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:57.207795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 527/1050] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 19:31:57.213514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ae76c-b6b6-4fa7-afd7-36bdf0d353ec None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:57.213761 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ae76c-b6b6-4fa7-afd7-36bdf0d353ec None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:57.214035 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ae76c-b6b6-4fa7-afd7-36bdf0d353ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:57.220664 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:57.234872 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-567ae76c-b6b6-4fa7-afd7-36bdf0d353ec None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:57.236355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-567ae76c-b6b6-4fa7-afd7-36bdf0d353ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:57.283466 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3d1a9-fc3c-4859-aae8-f8df73f20d4a None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:57.316335 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 528/1051] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 19:31:57.328048 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56c187-745f-4abf-a3fb-dd6f11f5b54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:57.328943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56c187-745f-4abf-a3fb-dd6f11f5b54d None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:57.329253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56c187-745f-4abf-a3fb-dd6f11f5b54d None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:57.329534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56c187-745f-4abf-a3fb-dd6f11f5b54d None admin] PATH_INFO: `/v3/users/640978855c9d4b14afe7d112957b80fb` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:57.341398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567ae76c-b6b6-4fa7-afd7-36bdf0d353ec None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:57.344135 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56c187-745f-4abf-a3fb-dd6f11f5b54d None admin] RBAC: Authorizing `identity:get_user(user_id=640978855c9d4b14afe7d112957b80fb)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:57.345759 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56c187-745f-4abf-a3fb-dd6f11f5b54d None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:57.348985 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 524/1052] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:31:57 2022] GET /identity/v3/users/640978855c9d4b14afe7d112957b80fb => generated 285 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:57.353901 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56c187-745f-4abf-a3fb-dd6f11f5b54d None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:57.463141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567ae76c-b6b6-4fa7-afd7-36bdf0d353ec None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:57.480648 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56c187-745f-4abf-a3fb-dd6f11f5b54d None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:57.492408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:57.494598 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:57.501232 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec070762-65b2-4c68-bf5b-1fd71fa828fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:57.504211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec070762-65b2-4c68-bf5b-1fd71fa828fe None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:57.504211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec070762-65b2-4c68-bf5b-1fd71fa828fe None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:57.504211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec070762-65b2-4c68-bf5b-1fd71fa828fe None admin] PATH_INFO: `/v3/services/image` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:57.504211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec070762-65b2-4c68-bf5b-1fd71fa828fe None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:57.505119 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec070762-65b2-4c68-bf5b-1fd71fa828fe None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:57.517414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-ec070762-65b2-4c68-bf5b-1fd71fa828fe None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 19:31:57.517414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:57.517414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:31:57.517414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:31:57.517414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:31:57.517414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:31:57.517414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:31:57.517414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:57.517414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 19:31:57.517414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:57.517414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:31:57.517414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:57.517414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:57.517414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:57.517414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:57.517414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:57.517414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:57.518467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:57.518467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:57.518467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:57.518467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:57.518467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:57.518467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:57.518467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:31:57.518467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:31:57.518467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:31:57.518467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:31:57.518467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:57.518467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:57.518467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:57.518467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:57.518467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:57.518467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:57.526498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:57.526498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:57.526498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:57.526498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:57.526498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:57.526498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:57.526498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:57.526498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:57.526498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:57.526498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:57.526498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:31:57.526498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:31:57.526498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 19:31:57.526498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:57.526498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 529/1053] 10.222.0.13 () {56 vars in 1105 bytes} [Fri May 27 19:31:57 2022] GET /identity/v3/services/image => generated 86 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:31:57.527916 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec070762-65b2-4c68-bf5b-1fd71fa828fe None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:57.534670 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ecd9f8-24c5-42b9-80d9-25515cef9d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:57.537122 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecd9f8-24c5-42b9-80d9-25515cef9d9a None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:57.537122 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecd9f8-24c5-42b9-80d9-25515cef9d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:57.537122 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ecd9f8-24c5-42b9-80d9-25515cef9d9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:57.623128 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ecd9f8-24c5-42b9-80d9-25515cef9d9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:57.624634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ecd9f8-24c5-42b9-80d9-25515cef9d9a None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:57.647836 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec070762-65b2-4c68-bf5b-1fd71fa828fe None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:57.690605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ed0c6-8df9-4327-a16f-21b48fc2fbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:57.691281 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ed0c6-8df9-4327-a16f-21b48fc2fbc7 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:57.697300 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ed0c6-8df9-4327-a16f-21b48fc2fbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:57.697300 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ed0c6-8df9-4327-a16f-21b48fc2fbc7 None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:57.697300 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ed0c6-8df9-4327-a16f-21b48fc2fbc7 None admin] RBAC: Adding query filter params (name=image) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:57.697300 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ed0c6-8df9-4327-a16f-21b48fc2fbc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:57.697300 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ed0c6-8df9-4327-a16f-21b48fc2fbc7 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:57.705004 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 530/1054] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:57.706226 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ed0c6-8df9-4327-a16f-21b48fc2fbc7 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:57.775819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 525/1055] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:31:57 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 19:31:57.775819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21023fb6-f7de-43b5-a90b-09512934e055 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:57.775819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21023fb6-f7de-43b5-a90b-09512934e055 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:57.775819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21023fb6-f7de-43b5-a90b-09512934e055 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:57.775819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 526/1056] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:31:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:31:57.777284 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21023fb6-f7de-43b5-a90b-09512934e055 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:57.789627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:57.792682 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:57.838330 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ed0c6-8df9-4327-a16f-21b48fc2fbc7 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:57.874749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75546ac-7f98-4d95-b735-ff18ed0a04ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:57.875530 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75546ac-7f98-4d95-b735-ff18ed0a04ce None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:57.875818 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75546ac-7f98-4d95-b735-ff18ed0a04ce None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:57.876101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75546ac-7f98-4d95-b735-ff18ed0a04ce None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:57.876815 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75546ac-7f98-4d95-b735-ff18ed0a04ce None admin] RBAC: Adding query filter params (type=image) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:57.877154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75546ac-7f98-4d95-b735-ff18ed0a04ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:57.878542 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75546ac-7f98-4d95-b735-ff18ed0a04ce None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:57.888560 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 531/1057] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:57 2022] GET /identity/v3/services?type=image => generated 350 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:57.888560 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a3ecf-f72a-4e91-a3ed-93935ecea8eb None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:57.888560 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a3ecf-f72a-4e91-a3ed-93935ecea8eb None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:57.888560 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a3ecf-f72a-4e91-a3ed-93935ecea8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:57.898547 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21023fb6-f7de-43b5-a90b-09512934e055 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:57.902626 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-3e4a3ecf-f72a-4e91-a3ed-93935ecea8eb None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:31:57.903825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-3e4a3ecf-f72a-4e91-a3ed-93935ecea8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:31:57.935144 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c564bf-7fa4-4dfc-821e-9d0e4c13fb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:57.936321 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c564bf-7fa4-4dfc-821e-9d0e4c13fb06 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:57.936604 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c564bf-7fa4-4dfc-821e-9d0e4c13fb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:57.936880 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c564bf-7fa4-4dfc-821e-9d0e4c13fb06 None admin] PATH_INFO: `/v3/projects/c0e4b22c01f241e9898771fd17fdd43d` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:57.938086 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c564bf-7fa4-4dfc-821e-9d0e4c13fb06 None admin] RBAC: Authorizing `identity:get_project(project_id=c0e4b22c01f241e9898771fd17fdd43d)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:57.939528 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c564bf-7fa4-4dfc-821e-9d0e4c13fb06 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:57.941291 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 527/1058] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:31:57 2022] GET /identity/v3/projects/c0e4b22c01f241e9898771fd17fdd43d => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:57.942077 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c564bf-7fa4-4dfc-821e-9d0e4c13fb06 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:57.974933 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 532/1059] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:31:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:31:57.975791 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a3ecf-f72a-4e91-a3ed-93935ecea8eb None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:58.028089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:58.031141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:58.050138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c564bf-7fa4-4dfc-821e-9d0e4c13fb06 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:58.086225 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99988d25-149c-4b6b-ae29-20a232b079be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:58.087468 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99988d25-149c-4b6b-ae29-20a232b079be None admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:58.088025 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99988d25-149c-4b6b-ae29-20a232b079be None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:58.088309 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99988d25-149c-4b6b-ae29-20a232b079be None admin] PATH_INFO: `/v3/endpoints` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:58.089106 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99988d25-149c-4b6b-ae29-20a232b079be None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:58.090489 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99988d25-149c-4b6b-ae29-20a232b079be None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:58.094483 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a3ecf-f72a-4e91-a3ed-93935ecea8eb None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:58.130444 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5ae55-83ba-470e-8654-845781edae60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:58.131114 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5ae55-83ba-470e-8654-845781edae60 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:58.131399 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5ae55-83ba-470e-8654-845781edae60 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:58.131677 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5ae55-83ba-470e-8654-845781edae60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:58.132551 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5ae55-83ba-470e-8654-845781edae60 None admin] RBAC: Adding query filter params (role.id=6505bbb129404436966ee59aa7a01d8e, scope.project.id=c0e4b22c01f241e9898771fd17fdd43d, user.id=640978855c9d4b14afe7d112957b80fb) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:58.132866 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5ae55-83ba-470e-8654-845781edae60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:58.134401 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a5ae55-83ba-470e-8654-845781edae60 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:58.140328 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 533/1060] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:31:57 2022] GET /identity/v3/role_assignments?user.id=640978855c9d4b14afe7d112957b80fb&scope.project.id=c0e4b22c01f241e9898771fd17fdd43d&role.id=6505bbb129404436966ee59aa7a01d8e => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:58.141242 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5ae55-83ba-470e-8654-845781edae60 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:58.201968 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 528/1061] 10.222.0.13 () {60 vars in 1144 bytes} [Fri May 27 19:31:57 2022] POST /identity/v3/endpoints => generated 327 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:31:58.276296 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5ae55-83ba-470e-8654-845781edae60 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:58.294810 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:58.331825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbcaba6-28b7-4c36-b3b6-25f93399a80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:58.332675 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcaba6-28b7-4c36-b3b6-25f93399a80d None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:58.332997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcaba6-28b7-4c36-b3b6-25f93399a80d None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:58.333323 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbcaba6-28b7-4c36-b3b6-25f93399a80d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:58.347496 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbcaba6-28b7-4c36-b3b6-25f93399a80d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:31:58.356901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-7cbcaba6-28b7-4c36-b3b6-25f93399a80d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 19:31:58.356901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:58.356901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:58.356901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:58.356901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:58.356901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:58.356901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:58.356901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:58.356901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:58.356901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:58.356901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:58.356901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:58.356901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:31:58.356901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:31:58.356901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:31:58.356901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:31:58.357942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:31:58.357942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:31:58.357942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:31:58.357942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:31:58.357942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:58.357942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:58.357942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:58.357942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:58.357942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:58.357942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:58.357942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:58.357942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:58.357942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:58.357942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:58.357942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:58.357942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:58.358807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:58.358807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:58.358807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:58.358807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:58.358807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:31:58.358807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:31:58.358807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:31:58.358807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:31:58.358807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:31:58.358807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:31:58.358807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 19:31:58.358807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:58.358807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 534/1062] 10.222.0.13 () {56 vars in 1120 bytes} [Fri May 27 19:31:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:58.364879 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99988d25-149c-4b6b-ae29-20a232b079be None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:58.493012 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99988d25-149c-4b6b-ae29-20a232b079be None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:58.535947 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434228e-d29a-4924-ae0c-38a8254c3f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:58.536630 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434228e-d29a-4924-ae0c-38a8254c3f00 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:58.536910 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434228e-d29a-4924-ae0c-38a8254c3f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:58.537210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e434228e-d29a-4924-ae0c-38a8254c3f00 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:58.537967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434228e-d29a-4924-ae0c-38a8254c3f00 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:58.538338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434228e-d29a-4924-ae0c-38a8254c3f00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:58.542806 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e434228e-d29a-4924-ae0c-38a8254c3f00 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:58.543327 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:58.559020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 529/1063] 10.222.0.13 () {56 vars in 1115 bytes} [Fri May 27 19:31:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:58.563720 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbcaba6-28b7-4c36-b3b6-25f93399a80d None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:58.688154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbcaba6-28b7-4c36-b3b6-25f93399a80d None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:58.734790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684333cd-3851-476d-b54b-91e296910aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:58.736085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684333cd-3851-476d-b54b-91e296910aae None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:58.736603 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684333cd-3851-476d-b54b-91e296910aae None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:58.737147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684333cd-3851-476d-b54b-91e296910aae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:58.751098 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684333cd-3851-476d-b54b-91e296910aae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:58.753229 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684333cd-3851-476d-b54b-91e296910aae None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:58.772135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-684333cd-3851-476d-b54b-91e296910aae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:58.772135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:58.772135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:58.772135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:58.772135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:58.772135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:58.772135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:58.772135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:58.772135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:58.772135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:58.772135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:58.772135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:58.772135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:58.772135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:58.772135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:58.773156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:58.773156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:58.773156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:58.773156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:58.773156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:58.773156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:58.773156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:58.773156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:58.773156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:58.773156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:58.773156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:58.773156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:58.773156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:58.773156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:58.773156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:58.773156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:58.773156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:58.774637 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:58.774637 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:58.774637 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:58.774637 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:58.780319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 535/1064] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:58.782179 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434228e-d29a-4924-ae0c-38a8254c3f00 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:58.898346 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e434228e-d29a-4924-ae0c-38a8254c3f00 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:58.938363 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8f0d3-786a-45d2-bbf0-593b219e9a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:58.939044 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8f0d3-786a-45d2-bbf0-593b219e9a79 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:58.939326 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8f0d3-786a-45d2-bbf0-593b219e9a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:58.939601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8f0d3-786a-45d2-bbf0-593b219e9a79 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:58.940623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8f0d3-786a-45d2-bbf0-593b219e9a79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:58.940623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8f0d3-786a-45d2-bbf0-593b219e9a79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:58.942180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8f0d3-786a-45d2-bbf0-593b219e9a79 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:58.960411 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 530/1065] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:58.962661 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684333cd-3851-476d-b54b-91e296910aae None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:58.988177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:59.072875 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684333cd-3851-476d-b54b-91e296910aae None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:59.110058 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be9934-fae0-4fcc-8278-bd70695682eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:59.110868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be9934-fae0-4fcc-8278-bd70695682eb None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:59.111289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be9934-fae0-4fcc-8278-bd70695682eb None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:59.111402 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be9934-fae0-4fcc-8278-bd70695682eb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:59.119776 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be9934-fae0-4fcc-8278-bd70695682eb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:59.121309 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95be9934-fae0-4fcc-8278-bd70695682eb None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:59.139268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-95be9934-fae0-4fcc-8278-bd70695682eb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 19:31:59.139268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:59.139268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:59.139268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:59.139268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:59.139268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:59.139268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:59.139268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:59.139268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:59.139268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:59.139268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:59.139268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:59.139268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:31:59.139268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:31:59.139268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:31:59.139268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:31:59.140667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:59.140667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:59.140667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:31:59.140667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:59.140667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:31:59.140667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:31:59.140667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:59.140667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:59.140667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:59.140667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:59.140667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:59.140667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:59.140667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:59.140667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:59.140667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:59.140667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:59.140667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:59.140667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:59.143059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:59.143059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:59.143059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:31:59.143059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:31:59.143059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:31:59.143059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:31:59.143059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:31:59.143059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:31:59.143059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 19:31:59.143059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:59.147679 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 536/1066] 10.222.0.13 () {56 vars in 1093 bytes} [Fri May 27 19:31:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:31:59.154573 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8f0d3-786a-45d2-bbf0-593b219e9a79 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:59.233885 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:59.303503 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8f0d3-786a-45d2-bbf0-593b219e9a79 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:59.348776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243348d-68e7-49d0-9e2e-3f38c58ace5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:59.349513 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243348d-68e7-49d0-9e2e-3f38c58ace5c None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:59.349787 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243348d-68e7-49d0-9e2e-3f38c58ace5c None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:59.350064 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243348d-68e7-49d0-9e2e-3f38c58ace5c None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:59.350913 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4243348d-68e7-49d0-9e2e-3f38c58ace5c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:59.351224 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4243348d-68e7-49d0-9e2e-3f38c58ace5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:59.353123 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4243348d-68e7-49d0-9e2e-3f38c58ace5c None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:59.377233 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 531/1067] 10.222.0.13 () {56 vars in 1133 bytes} [Fri May 27 19:31:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:59.378902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be9934-fae0-4fcc-8278-bd70695682eb None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:59.504777 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:59.534200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be9934-fae0-4fcc-8278-bd70695682eb None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:59.589279 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e45a03-88a7-4383-8f71-2b8ad821eed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:59.590709 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e45a03-88a7-4383-8f71-2b8ad821eed5 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:59.591216 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e45a03-88a7-4383-8f71-2b8ad821eed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:59.591752 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e45a03-88a7-4383-8f71-2b8ad821eed5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:59.603693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e45a03-88a7-4383-8f71-2b8ad821eed5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:59.605213 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e45a03-88a7-4383-8f71-2b8ad821eed5 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:59.636268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-b3e45a03-88a7-4383-8f71-2b8ad821eed5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:59.636268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:31:59.636268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:31:59.636268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:31:59.636268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:31:59.636268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:31:59.636268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:31:59.636268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:31:59.636268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:31:59.636268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:31:59.636268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:31:59.636268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:31:59.636268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:31:59.636268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:31:59.636268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:31:59.637310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:31:59.637310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:31:59.637310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:31:59.637310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:31:59.637310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:31:59.637310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:31:59.637310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:31:59.637310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:31:59.637310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:31:59.637310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:31:59.637310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:31:59.637310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:31:59.637310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:31:59.637310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:31:59.637310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:31:59.637310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:31:59.637310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:31:59.639299 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:31:59.639299 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:31:59.639299 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:31:59.639299 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:31:59.639299 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 537/1068] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:31:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:31:59.643280 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243348d-68e7-49d0-9e2e-3f38c58ace5c None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:59.775127 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4243348d-68e7-49d0-9e2e-3f38c58ace5c None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:31:59.793734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:31:59.817869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179bcf1-f40b-4293-9cdc-f302bb49b977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:31:59.818548 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179bcf1-f40b-4293-9cdc-f302bb49b977 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:31:59.818832 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179bcf1-f40b-4293-9cdc-f302bb49b977 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:31:59.819135 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3179bcf1-f40b-4293-9cdc-f302bb49b977 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:31:59.819991 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179bcf1-f40b-4293-9cdc-f302bb49b977 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:31:59.820297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179bcf1-f40b-4293-9cdc-f302bb49b977 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:31:59.822500 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3179bcf1-f40b-4293-9cdc-f302bb49b977 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:31:59.838953 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 532/1069] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:31:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:31:59.844619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e45a03-88a7-4383-8f71-2b8ad821eed5 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:31:59.969395 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e45a03-88a7-4383-8f71-2b8ad821eed5 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:00.017500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:00.025103 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ecb400-df95-4a4b-a848-a4324704ac29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:00.025997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ecb400-df95-4a4b-a848-a4324704ac29 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:00.026207 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ecb400-df95-4a4b-a848-a4324704ac29 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:00.026476 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ecb400-df95-4a4b-a848-a4324704ac29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:00.034022 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ecb400-df95-4a4b-a848-a4324704ac29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:00.037228 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ecb400-df95-4a4b-a848-a4324704ac29 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:00.053435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-06ecb400-df95-4a4b-a848-a4324704ac29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:32:00.053435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:32:00.053435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:32:00.053435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:32:00.053435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:32:00.053435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:32:00.053435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:32:00.053435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:32:00.053435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:32:00.053435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:32:00.053435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:32:00.053435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:32:00.053435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:32:00.053435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:32:00.053435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:32:00.054968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:32:00.054968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:32:00.054968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:32:00.054968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:32:00.054968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:32:00.054968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:32:00.054968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:32:00.054968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:32:00.054968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:32:00.054968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:32:00.054968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:32:00.054968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:32:00.054968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:32:00.054968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:32:00.054968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:32:00.054968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:32:00.054968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:32:00.056520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:32:00.056520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:32:00.056520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:32:00.056520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:32:00.056520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:32:00.056520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:32:00.056520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:32:00.056520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:32:00.056520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 538/1070] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:31:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:32:00.064129 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179bcf1-f40b-4293-9cdc-f302bb49b977 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:00.177663 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3179bcf1-f40b-4293-9cdc-f302bb49b977 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:00.216371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e757d-e108-41ec-9e85-77b1c01e1c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:00.217267 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e757d-e108-41ec-9e85-77b1c01e1c77 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:00.217689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e757d-e108-41ec-9e85-77b1c01e1c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:00.218097 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e757d-e108-41ec-9e85-77b1c01e1c77 None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:00.219061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e757d-e108-41ec-9e85-77b1c01e1c77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:00.219507 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e757d-e108-41ec-9e85-77b1c01e1c77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:00.221268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e757d-e108-41ec-9e85-77b1c01e1c77 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:00.241051 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 533/1071] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:32:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:00.244405 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ecb400-df95-4a4b-a848-a4324704ac29 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:00.368683 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ecb400-df95-4a4b-a848-a4324704ac29 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:00.409342 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d8ecc-c70f-49ee-bc22-99b0bd25f216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:00.410238 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d8ecc-c70f-49ee-bc22-99b0bd25f216 None admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:00.410616 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d8ecc-c70f-49ee-bc22-99b0bd25f216 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:00.410798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d8ecc-c70f-49ee-bc22-99b0bd25f216 None admin] PATH_INFO: `/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/7542e1fd160e4b2c9d25accf19aad19f/roles/30c027fab0344b47b1ba1ef6c6cbd818` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:00.413034 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d8ecc-c70f-49ee-bc22-99b0bd25f216 None admin] RBAC: Authorizing `identity:create_grant(project_id=a47d6b6971c3452e9799e8146856bdc5, user_id=7542e1fd160e4b2c9d25accf19aad19f, role_id=30c027fab0344b47b1ba1ef6c6cbd818)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:00.415021 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d8ecc-c70f-49ee-bc22-99b0bd25f216 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:00.493941 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 539/1072] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:32:00 2022] PUT /identity/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/7542e1fd160e4b2c9d25accf19aad19f/roles/30c027fab0344b47b1ba1ef6c6cbd818 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:32:00.573172 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e338e5f-ac63-437e-bcfe-c18b174475c4 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:00.573485 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e338e5f-ac63-437e-bcfe-c18b174475c4 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:00.573939 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e338e5f-ac63-437e-bcfe-c18b174475c4 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:00.575848 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 534/1073] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:32:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:32:00.582302 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b910df9-4678-46ce-8979-253b308cd72a None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:00.582628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b910df9-4678-46ce-8979-253b308cd72a None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:00.582866 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b910df9-4678-46ce-8979-253b308cd72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:00.603295 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-0b910df9-4678-46ce-8979-253b308cd72a None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:32:00.604562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-0b910df9-4678-46ce-8979-253b308cd72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:32:00.662915 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2380e4-eabc-4bb6-94c2-e9ff070644b2 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:00.663095 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2380e4-eabc-4bb6-94c2-e9ff070644b2 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:00.663371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2380e4-eabc-4bb6-94c2-e9ff070644b2 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:00.664652 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:00.664852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 535/1074] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:32:00.670961 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8961213-bb7a-492f-8865-fb284745bb33 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:00.671300 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8961213-bb7a-492f-8865-fb284745bb33 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:00.671585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8961213-bb7a-492f-8865-fb284745bb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:00.689333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 540/1075] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:32:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:32:00.689744 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-c8961213-bb7a-492f-8865-fb284745bb33 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:32:00.690969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-c8961213-bb7a-492f-8865-fb284745bb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:32:00.713866 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b910df9-4678-46ce-8979-253b308cd72a None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:00.733739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:00.777958 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 536/1076] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:32:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 19:32:00.800655 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8961213-bb7a-492f-8865-fb284745bb33 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:00.851267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b910df9-4678-46ce-8979-253b308cd72a None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:00.897194 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aaf594-67a6-4a19-967e-1bb62abb9ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:00.898003 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aaf594-67a6-4a19-967e-1bb62abb9ffa None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:00.898284 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aaf594-67a6-4a19-967e-1bb62abb9ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:00.899648 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aaf594-67a6-4a19-967e-1bb62abb9ffa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:00.910752 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49aaf594-67a6-4a19-967e-1bb62abb9ffa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:32:00.912180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8961213-bb7a-492f-8865-fb284745bb33 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:00.918628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-49aaf594-67a6-4a19-967e-1bb62abb9ffa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:32:00.918628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:32:00.918628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:32:00.918628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:32:00.918628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:32:00.918628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:32:00.918628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:32:00.918628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:32:00.918628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:32:00.918628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:32:00.918628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:32:00.918628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:32:00.918628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:32:00.918628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:32:00.918628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:32:00.918628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:32:00.919711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:32:00.919711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:32:00.919711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:32:00.919711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:32:00.919711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:32:00.919711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:32:00.919711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:32:00.919711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:32:00.919711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:32:00.919711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:32:00.919711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:32:00.919711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:32:00.919711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:32:00.919711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:32:00.919711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:32:00.919711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:32:00.922876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:32:00.922876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:32:00.922876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:32:00.922876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:32:00.922876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:32:00.922876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:32:00.922876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:32:00.922876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:32:00.922876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:32:00.922876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:32:00.922876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:32:00.922876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:32:00.922876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 541/1077] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:32:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:32:00.930266 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aaf594-67a6-4a19-967e-1bb62abb9ffa None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:00.951433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba09d3-5d82-4886-aa67-08f057cbab63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:00.952106 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba09d3-5d82-4886-aa67-08f057cbab63 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:00.952382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba09d3-5d82-4886-aa67-08f057cbab63 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:00.952655 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba09d3-5d82-4886-aa67-08f057cbab63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:00.982428 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:00.987979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:01.040853 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba09d3-5d82-4886-aa67-08f057cbab63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:01.043083 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba09d3-5d82-4886-aa67-08f057cbab63 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:01.058046 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aaf594-67a6-4a19-967e-1bb62abb9ffa None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:01.098196 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa88a4-73b6-46ea-9c16-bc6ca24dacc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:01.098862 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa88a4-73b6-46ea-9c16-bc6ca24dacc7 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:01.099147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa88a4-73b6-46ea-9c16-bc6ca24dacc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:01.099424 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa88a4-73b6-46ea-9c16-bc6ca24dacc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:01.100167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa88a4-73b6-46ea-9c16-bc6ca24dacc7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:01.100472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa88a4-73b6-46ea-9c16-bc6ca24dacc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:01.102816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa88a4-73b6-46ea-9c16-bc6ca24dacc7 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:01.112050 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 542/1078] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:32:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:01.117806 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa88a4-73b6-46ea-9c16-bc6ca24dacc7 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:01.160496 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 537/1079] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:32:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 19:32:01.161701 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba09d3-5d82-4886-aa67-08f057cbab63 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:01.229163 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:01.231679 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa88a4-73b6-46ea-9c16-bc6ca24dacc7 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:01.235212 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:01.267513 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba09d3-5d82-4886-aa67-08f057cbab63 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:01.268515 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2513713-4671-44a7-9fc0-df7450d34518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:01.269421 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2513713-4671-44a7-9fc0-df7450d34518 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:01.269707 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2513713-4671-44a7-9fc0-df7450d34518 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:01.269985 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2513713-4671-44a7-9fc0-df7450d34518 None admin] PATH_INFO: `/v3/users/640978855c9d4b14afe7d112957b80fb` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:01.271219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2513713-4671-44a7-9fc0-df7450d34518 None admin] RBAC: Authorizing `identity:get_user(user_id=640978855c9d4b14afe7d112957b80fb)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:01.272705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2513713-4671-44a7-9fc0-df7450d34518 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:01.274868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 543/1080] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:32:01 2022] GET /identity/v3/users/640978855c9d4b14afe7d112957b80fb => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:32:01.280563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2513713-4671-44a7-9fc0-df7450d34518 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:01.311819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aac567-4995-46e4-8eff-1e79cc96119f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:01.312974 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aac567-4995-46e4-8eff-1e79cc96119f None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:01.313445 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aac567-4995-46e4-8eff-1e79cc96119f None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:01.313894 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aac567-4995-46e4-8eff-1e79cc96119f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:01.332892 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aac567-4995-46e4-8eff-1e79cc96119f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:01.334404 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86aac567-4995-46e4-8eff-1e79cc96119f None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:01.356653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-86aac567-4995-46e4-8eff-1e79cc96119f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:32:01.356653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:32:01.356653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:32:01.356653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:32:01.356653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:32:01.356653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:32:01.356653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:32:01.356653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:32:01.356653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:32:01.356653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:32:01.356653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:32:01.356653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:32:01.356653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:32:01.356653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:32:01.356653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:32:01.357680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:32:01.357680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:32:01.357680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:32:01.357680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:32:01.357680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:32:01.357680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:32:01.357680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:32:01.357680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:32:01.357680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:32:01.357680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:32:01.357680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:32:01.357680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:32:01.357680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:32:01.357680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:32:01.357680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:32:01.357680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:32:01.357680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:32:01.358599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:32:01.358599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:32:01.358599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:32:01.358599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:32:01.358599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 538/1081] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:32:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:32:01.365512 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aac567-4995-46e4-8eff-1e79cc96119f None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:01.410667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2513713-4671-44a7-9fc0-df7450d34518 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:01.453083 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977bb98-ea4f-455d-8ebe-43ff4246bcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:01.454083 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977bb98-ea4f-455d-8ebe-43ff4246bcdc None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:01.454449 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977bb98-ea4f-455d-8ebe-43ff4246bcdc None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:01.454653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c977bb98-ea4f-455d-8ebe-43ff4246bcdc None admin] PATH_INFO: `/v3/projects/c0e4b22c01f241e9898771fd17fdd43d` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:01.456081 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c977bb98-ea4f-455d-8ebe-43ff4246bcdc None admin] RBAC: Authorizing `identity:get_project(project_id=c0e4b22c01f241e9898771fd17fdd43d)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:01.458296 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c977bb98-ea4f-455d-8ebe-43ff4246bcdc None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:01.459474 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 544/1082] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:32:01 2022] GET /identity/v3/projects/c0e4b22c01f241e9898771fd17fdd43d => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:32:01.466070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977bb98-ea4f-455d-8ebe-43ff4246bcdc None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:01.491149 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86aac567-4995-46e4-8eff-1e79cc96119f None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:01.511403 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:01.515967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:01.541252 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38625ed7-1a50-488d-885b-9acef2f7e202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:01.541963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38625ed7-1a50-488d-885b-9acef2f7e202 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:01.542252 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38625ed7-1a50-488d-885b-9acef2f7e202 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:01.543172 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38625ed7-1a50-488d-885b-9acef2f7e202 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:01.543285 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38625ed7-1a50-488d-885b-9acef2f7e202 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:01.543651 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38625ed7-1a50-488d-885b-9acef2f7e202 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:01.545038 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38625ed7-1a50-488d-885b-9acef2f7e202 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:01.558489 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 539/1083] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:32:01.579108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c977bb98-ea4f-455d-8ebe-43ff4246bcdc None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:01.621217 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81411dc5-83b4-41f9-b360-631791838d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:01.622139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81411dc5-83b4-41f9-b360-631791838d5d None admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:01.622418 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81411dc5-83b4-41f9-b360-631791838d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:01.622689 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81411dc5-83b4-41f9-b360-631791838d5d None admin] PATH_INFO: `/v3/projects/c0e4b22c01f241e9898771fd17fdd43d/users/640978855c9d4b14afe7d112957b80fb/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:01.624799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81411dc5-83b4-41f9-b360-631791838d5d None admin] RBAC: Authorizing `identity:create_grant(project_id=c0e4b22c01f241e9898771fd17fdd43d, user_id=640978855c9d4b14afe7d112957b80fb, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:01.626312 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81411dc5-83b4-41f9-b360-631791838d5d None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:01.715673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 545/1084] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:32:01 2022] PUT /identity/v3/projects/c0e4b22c01f241e9898771fd17fdd43d/users/640978855c9d4b14afe7d112957b80fb/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:32:02.843391 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910fd22-beda-44ac-acad-806b4917f996 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:02.843927 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910fd22-beda-44ac-acad-806b4917f996 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:02.844020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2910fd22-beda-44ac-acad-806b4917f996 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:02.845678 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 540/1085] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:32:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:32:02.851833 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c661622-d3ea-4dce-be57-569526f95a54 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:02.851833 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c661622-d3ea-4dce-be57-569526f95a54 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:02.853064 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c661622-d3ea-4dce-be57-569526f95a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:02.870274 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-6c661622-d3ea-4dce-be57-569526f95a54 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:32:02.873073 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-6c661622-d3ea-4dce-be57-569526f95a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:32:02.943791 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 546/1086] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:32:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:32:02.969438 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910fd22-beda-44ac-acad-806b4917f996 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:03.075597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2910fd22-beda-44ac-acad-806b4917f996 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:03.109045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffc6b1-faa9-44e2-95c4-f3987a77ff98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:03.109777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffc6b1-faa9-44e2-95c4-f3987a77ff98 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:03.110061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffc6b1-faa9-44e2-95c4-f3987a77ff98 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:03.110334 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffc6b1-faa9-44e2-95c4-f3987a77ff98 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:03.122148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffc6b1-faa9-44e2-95c4-f3987a77ff98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:32:03.130196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-cbffc6b1-faa9-44e2-95c4-f3987a77ff98 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 19:32:03.130196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:32:03.130196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:32:03.130196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:32:03.130196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:32:03.130196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:32:03.130196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:32:03.130196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:32:03.130196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:32:03.130196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:32:03.130196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:32:03.130196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:32:03.130196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:32:03.130196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:32:03.130196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:32:03.130196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:32:03.131371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:32:03.131371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:32:03.131371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:32:03.131371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:32:03.131371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:32:03.131371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:32:03.131371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:32:03.131371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:32:03.131371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:32:03.131371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:32:03.131371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:32:03.131371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:32:03.131371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:32:03.131371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:32:03.131371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:32:03.131371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:32:03.132255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:32:03.132255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:32:03.132255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:32:03.132255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:32:03.132255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:32:03.132255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:32:03.132255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:32:03.132255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:32:03.132255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:32:03.132255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:32:03.132255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 19:32:03.132255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:32:03.132255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 541/1087] 10.222.0.13 () {56 vars in 1120 bytes} [Fri May 27 19:32:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:32:03.138532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c661622-d3ea-4dce-be57-569526f95a54 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:03.167975 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:03.241564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c661622-d3ea-4dce-be57-569526f95a54 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:03.276509 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e7ccd-2d30-4e81-b24d-742a719406f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:03.277206 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e7ccd-2d30-4e81-b24d-742a719406f8 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:03.277490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e7ccd-2d30-4e81-b24d-742a719406f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:03.277768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e7ccd-2d30-4e81-b24d-742a719406f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:03.278506 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e7ccd-2d30-4e81-b24d-742a719406f8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:03.278855 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e7ccd-2d30-4e81-b24d-742a719406f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:03.280400 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e7ccd-2d30-4e81-b24d-742a719406f8 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:03.290661 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 547/1088] 10.222.0.13 () {56 vars in 1115 bytes} [Fri May 27 19:32:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:03.296309 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffc6b1-faa9-44e2-95c4-f3987a77ff98 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:03.333601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:03.407679 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffc6b1-faa9-44e2-95c4-f3987a77ff98 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:03.443088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9342cfa-9fc4-4fa0-87ce-b206b44e077f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:03.443892 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9342cfa-9fc4-4fa0-87ce-b206b44e077f None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:03.444176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9342cfa-9fc4-4fa0-87ce-b206b44e077f None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:03.444455 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9342cfa-9fc4-4fa0-87ce-b206b44e077f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:03.455791 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9342cfa-9fc4-4fa0-87ce-b206b44e077f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:03.457405 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9342cfa-9fc4-4fa0-87ce-b206b44e077f None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:03.476034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-f9342cfa-9fc4-4fa0-87ce-b206b44e077f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:32:03.476034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:32:03.476034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:32:03.476034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:32:03.476034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:32:03.476034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:32:03.476034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:32:03.476034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:32:03.476034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:32:03.476034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:32:03.476034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:32:03.476034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:32:03.476034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:32:03.476034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:32:03.476034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:32:03.477114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:32:03.477114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:32:03.477114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:32:03.477114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:32:03.477114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:32:03.477114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:32:03.477114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:32:03.477114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:32:03.477114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:32:03.477114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:32:03.477114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:32:03.477114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:32:03.477114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:32:03.477114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:32:03.477114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:32:03.477114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:32:03.477114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:32:03.478062 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:32:03.478062 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:32:03.478062 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:32:03.478062 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:32:03.478062 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 542/1089] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:32:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:32:03.483163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e7ccd-2d30-4e81-b24d-742a719406f8 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:03.577184 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:03.586454 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e7ccd-2d30-4e81-b24d-742a719406f8 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:03.620841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a9f4a-e5c9-4973-8d3e-8b99ff2524c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:03.621596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a9f4a-e5c9-4973-8d3e-8b99ff2524c9 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:03.621870 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a9f4a-e5c9-4973-8d3e-8b99ff2524c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:03.622139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a9f4a-e5c9-4973-8d3e-8b99ff2524c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:03.622850 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a9f4a-e5c9-4973-8d3e-8b99ff2524c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:03.623153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a9f4a-e5c9-4973-8d3e-8b99ff2524c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:03.624523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a9f4a-e5c9-4973-8d3e-8b99ff2524c9 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:03.639457 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 548/1090] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:32:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:03.645068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9342cfa-9fc4-4fa0-87ce-b206b44e077f None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:03.733910 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:03.746569 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9342cfa-9fc4-4fa0-87ce-b206b44e077f None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:03.780289 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1939b-ebbc-4150-9f9f-d49a960f25da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:03.781113 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1939b-ebbc-4150-9f9f-d49a960f25da None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:03.781401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1939b-ebbc-4150-9f9f-d49a960f25da None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:03.781678 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f1939b-ebbc-4150-9f9f-d49a960f25da None admin] PATH_INFO: `/v3/users/nova` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:03.789296 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f1939b-ebbc-4150-9f9f-d49a960f25da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:03.791039 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f1939b-ebbc-4150-9f9f-d49a960f25da None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:03.806949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-e2f1939b-ebbc-4150-9f9f-d49a960f25da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 19:32:03.806949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:32:03.806949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:32:03.806949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:32:03.806949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:32:03.806949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:32:03.806949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:32:03.806949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:32:03.806949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:32:03.806949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:32:03.806949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:32:03.806949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:32:03.806949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:32:03.806949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:32:03.806949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:32:03.806949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:32:03.808032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:32:03.808032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:32:03.808032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:32:03.808032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:32:03.808032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:32:03.808032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:32:03.808032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:32:03.808032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:32:03.808032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:32:03.808032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:32:03.808032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:32:03.808032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:32:03.808032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:32:03.808032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:32:03.808032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:32:03.808032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:32:03.808032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:32:03.808032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:32:03.809071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:32:03.809071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:32:03.809071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:32:03.809071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:32:03.809071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:32:03.809071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:32:03.809071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:32:03.809071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:32:03.809071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 19:32:03.809071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:32:03.809071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 543/1091] 10.222.0.13 () {56 vars in 1093 bytes} [Fri May 27 19:32:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:32:03.814514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a9f4a-e5c9-4973-8d3e-8b99ff2524c9 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:03.918796 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a9f4a-e5c9-4973-8d3e-8b99ff2524c9 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:03.952725 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd241f6-b2a8-48df-8cce-1fe42b9bd7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:03.953422 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd241f6-b2a8-48df-8cce-1fe42b9bd7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:03.953737 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd241f6-b2a8-48df-8cce-1fe42b9bd7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:03.954210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd241f6-b2a8-48df-8cce-1fe42b9bd7d7 None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:03.954860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd241f6-b2a8-48df-8cce-1fe42b9bd7d7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:03.955166 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd241f6-b2a8-48df-8cce-1fe42b9bd7d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:03.956621 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd241f6-b2a8-48df-8cce-1fe42b9bd7d7 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:03.974536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 549/1092] 10.222.0.13 () {56 vars in 1133 bytes} [Fri May 27 19:32:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:03.980567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1939b-ebbc-4150-9f9f-d49a960f25da None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:04.003937 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b3164-257f-4caa-b258-f237bee83366 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:04.004154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b3164-257f-4caa-b258-f237bee83366 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:04.004599 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b3164-257f-4caa-b258-f237bee83366 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:04.006014 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 550/1093] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:32:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 19:32:04.011809 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb5b5b-b8af-410d-9e94-92330fa7b016 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:04.012110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb5b5b-b8af-410d-9e94-92330fa7b016 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:04.012393 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb5b5b-b8af-410d-9e94-92330fa7b016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:04.028978 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-b7cb5b5b-b8af-410d-9e94-92330fa7b016 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:32:04.030177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-b7cb5b5b-b8af-410d-9e94-92330fa7b016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:32:04.032928 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:04.083250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f1939b-ebbc-4150-9f9f-d49a960f25da None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:04.106103 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 551/1094] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:32:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 19:32:04.120800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f1837-a4bc-457a-b5ea-b913441b519d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:04.121492 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f1837-a4bc-457a-b5ea-b913441b519d None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:04.121775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f1837-a4bc-457a-b5ea-b913441b519d None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:04.122051 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897f1837-a4bc-457a-b5ea-b913441b519d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:04.130964 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb5b5b-b8af-410d-9e94-92330fa7b016 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:04.142005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:04.193813 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f1837-a4bc-457a-b5ea-b913441b519d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:04.196407 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897f1837-a4bc-457a-b5ea-b913441b519d None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:04.235732 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb5b5b-b8af-410d-9e94-92330fa7b016 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:04.271742 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881b9af-f27a-41f0-b6a1-f34576313fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:04.272460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881b9af-f27a-41f0-b6a1-f34576313fad None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:04.272747 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881b9af-f27a-41f0-b6a1-f34576313fad None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:04.273087 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881b9af-f27a-41f0-b6a1-f34576313fad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:04.281082 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:04.284411 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881b9af-f27a-41f0-b6a1-f34576313fad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:32:04.293086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-2881b9af-f27a-41f0-b6a1-f34576313fad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:32:04.293086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:32:04.293086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:32:04.293086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:32:04.293086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:32:04.293086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:32:04.293086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:32:04.293086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:32:04.293086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:32:04.293086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:32:04.293086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:32:04.293086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:32:04.293086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:32:04.293086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:32:04.293086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:32:04.293086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:32:04.294138 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:32:04.294138 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:32:04.294138 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:32:04.294138 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:32:04.294138 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:32:04.294138 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:32:04.294138 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:32:04.294138 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:32:04.294138 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:32:04.294138 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:32:04.294138 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:32:04.294138 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:32:04.294138 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:32:04.294138 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:32:04.294138 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:32:04.294138 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:32:04.295097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:32:04.295097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:32:04.295097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:32:04.295097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:32:04.295097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:32:04.295097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:32:04.295097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:32:04.295097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:32:04.295097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:32:04.295097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:32:04.295097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:32:04.295097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:32:04.295097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 552/1095] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:32:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:32:04.300643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881b9af-f27a-41f0-b6a1-f34576313fad None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:04.303264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 544/1096] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:32:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 19:32:04.309538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f1837-a4bc-457a-b5ea-b913441b519d None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:04.360896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:04.407678 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881b9af-f27a-41f0-b6a1-f34576313fad None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:04.416750 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897f1837-a4bc-457a-b5ea-b913441b519d None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:04.441610 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f92a1-d9d7-4ce9-83cd-0bfd3be60b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:04.442288 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f92a1-d9d7-4ce9-83cd-0bfd3be60b6f None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:04.442578 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f92a1-d9d7-4ce9-83cd-0bfd3be60b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:04.442858 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f92a1-d9d7-4ce9-83cd-0bfd3be60b6f None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:04.443609 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f92a1-d9d7-4ce9-83cd-0bfd3be60b6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:04.443919 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f92a1-d9d7-4ce9-83cd-0bfd3be60b6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:04.445865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f92a1-d9d7-4ce9-83cd-0bfd3be60b6f None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:04.452569 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c786be03-66aa-41cc-a651-1c7241d93236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:04.453399 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786be03-66aa-41cc-a651-1c7241d93236 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:04.453690 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786be03-66aa-41cc-a651-1c7241d93236 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:04.453969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786be03-66aa-41cc-a651-1c7241d93236 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:04.457367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 553/1097] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:32:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:32:04.463897 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f92a1-d9d7-4ce9-83cd-0bfd3be60b6f None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:04.465396 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c786be03-66aa-41cc-a651-1c7241d93236 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:04.466860 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c786be03-66aa-41cc-a651-1c7241d93236 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:04.485518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-c786be03-66aa-41cc-a651-1c7241d93236 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:32:04.485518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:32:04.485518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:32:04.485518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:32:04.485518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:32:04.485518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:32:04.485518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:32:04.485518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:32:04.485518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:32:04.485518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:32:04.485518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:32:04.485518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:32:04.485518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:32:04.485518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:32:04.485518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:32:04.486638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:32:04.486638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:32:04.486638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:32:04.486638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:32:04.486638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:32:04.486638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:32:04.486638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:32:04.486638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:32:04.486638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:32:04.486638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:32:04.486638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:32:04.486638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:32:04.486638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:32:04.486638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:32:04.486638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:32:04.486638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:32:04.486638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:32:04.487635 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:32:04.487635 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:32:04.487635 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:32:04.487635 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:32:04.487635 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 545/1098] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:32:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:32:04.492905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c786be03-66aa-41cc-a651-1c7241d93236 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:04.532116 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:04.572145 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f92a1-d9d7-4ce9-83cd-0bfd3be60b6f None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:04.591883 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:04.594827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c786be03-66aa-41cc-a651-1c7241d93236 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:04.606633 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aed695-2cc7-4ff7-95b5-edf78f42d491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:04.607542 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aed695-2cc7-4ff7-95b5-edf78f42d491 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:04.607824 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aed695-2cc7-4ff7-95b5-edf78f42d491 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:04.608102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aed695-2cc7-4ff7-95b5-edf78f42d491 None admin] PATH_INFO: `/v3/users/640978855c9d4b14afe7d112957b80fb` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:04.609428 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aed695-2cc7-4ff7-95b5-edf78f42d491 None admin] RBAC: Authorizing `identity:get_user(user_id=640978855c9d4b14afe7d112957b80fb)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:04.610953 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aed695-2cc7-4ff7-95b5-edf78f42d491 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:04.612595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 554/1099] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:32:04 2022] GET /identity/v3/users/640978855c9d4b14afe7d112957b80fb => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:32:04.619157 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aed695-2cc7-4ff7-95b5-edf78f42d491 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:04.630230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216badbe-14b8-4ad4-9d07-15a8fffdadb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:04.630911 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216badbe-14b8-4ad4-9d07-15a8fffdadb7 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:04.631200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216badbe-14b8-4ad4-9d07-15a8fffdadb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:04.631780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216badbe-14b8-4ad4-9d07-15a8fffdadb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:04.633150 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216badbe-14b8-4ad4-9d07-15a8fffdadb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:04.633670 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216badbe-14b8-4ad4-9d07-15a8fffdadb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:04.635063 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216badbe-14b8-4ad4-9d07-15a8fffdadb7 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:04.646634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 546/1100] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:32:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:32:04.652486 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216badbe-14b8-4ad4-9d07-15a8fffdadb7 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:04.727117 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aed695-2cc7-4ff7-95b5-edf78f42d491 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:04.755708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:04.757066 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216badbe-14b8-4ad4-9d07-15a8fffdadb7 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:04.762452 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcd58d-583c-4c16-b838-5ed4c59198bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:04.763107 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcd58d-583c-4c16-b838-5ed4c59198bb None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:04.763386 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcd58d-583c-4c16-b838-5ed4c59198bb None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:04.763660 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcd58d-583c-4c16-b838-5ed4c59198bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:04.792058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26e010-6a66-41b4-b8af-a39fe80c69bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:04.792780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26e010-6a66-41b4-b8af-a39fe80c69bf None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:04.793102 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26e010-6a66-41b4-b8af-a39fe80c69bf None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:04.793382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26e010-6a66-41b4-b8af-a39fe80c69bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:04.800032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26e010-6a66-41b4-b8af-a39fe80c69bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:04.801515 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e26e010-6a66-41b4-b8af-a39fe80c69bf None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:04.816069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-7e26e010-6a66-41b4-b8af-a39fe80c69bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:32:04.816069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:32:04.816069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:32:04.816069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:32:04.816069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:32:04.816069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:32:04.816069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:32:04.816069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:32:04.816069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:32:04.816069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:32:04.816069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:32:04.816069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:32:04.816069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:32:04.816069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:32:04.816069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:32:04.817368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:32:04.817368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:32:04.817368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:32:04.817368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:32:04.817368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:32:04.817368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:32:04.817368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:32:04.817368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:32:04.817368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:32:04.817368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:32:04.817368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:32:04.817368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:32:04.817368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:32:04.817368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:32:04.817368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:32:04.817368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:32:04.817368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:32:04.818735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:32:04.818735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:32:04.818735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:32:04.818735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:32:04.818735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:32:04.818735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:32:04.818735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:32:04.818735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:32:04.818735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 547/1101] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:32:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:32:04.824539 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26e010-6a66-41b4-b8af-a39fe80c69bf None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:04.833179 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bcd58d-583c-4c16-b838-5ed4c59198bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:04.834670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bcd58d-583c-4c16-b838-5ed4c59198bb None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:04.847141 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:04.926534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26e010-6a66-41b4-b8af-a39fe80c69bf None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:04.934025 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 555/1102] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:32:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 19:32:04.939887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcd58d-583c-4c16-b838-5ed4c59198bb None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:04.964655 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaec24b-c710-48db-9122-c5029fa8c00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:04.965618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaec24b-c710-48db-9122-c5029fa8c00c None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:04.965902 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaec24b-c710-48db-9122-c5029fa8c00c None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:04.966180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaec24b-c710-48db-9122-c5029fa8c00c None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:04.967071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaec24b-c710-48db-9122-c5029fa8c00c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:04.967306 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaec24b-c710-48db-9122-c5029fa8c00c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:04.968780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaec24b-c710-48db-9122-c5029fa8c00c None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:04.970676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:04.981304 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 548/1103] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:32:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:32:04.987509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaec24b-c710-48db-9122-c5029fa8c00c None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:05.047586 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcd58d-583c-4c16-b838-5ed4c59198bb None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:05.075222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:05.084081 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905e017-ae06-4730-8363-67f644774bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:05.088325 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905e017-ae06-4730-8363-67f644774bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:05.088325 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905e017-ae06-4730-8363-67f644774bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:05.088325 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0905e017-ae06-4730-8363-67f644774bd0 None admin] PATH_INFO: `/v3/projects/c0e4b22c01f241e9898771fd17fdd43d` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:05.088325 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905e017-ae06-4730-8363-67f644774bd0 None admin] RBAC: Authorizing `identity:get_project(project_id=c0e4b22c01f241e9898771fd17fdd43d)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:05.089703 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0905e017-ae06-4730-8363-67f644774bd0 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:05.096600 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 556/1104] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:32:04 2022] GET /identity/v3/projects/c0e4b22c01f241e9898771fd17fdd43d => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:32:05.096772 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaec24b-c710-48db-9122-c5029fa8c00c None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:05.097424 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905e017-ae06-4730-8363-67f644774bd0 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:05.129200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828adc35-d211-4f9e-8a65-15dd4a56d9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:05.129860 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828adc35-d211-4f9e-8a65-15dd4a56d9cf None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:05.130144 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828adc35-d211-4f9e-8a65-15dd4a56d9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:05.130422 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828adc35-d211-4f9e-8a65-15dd4a56d9cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:05.131247 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828adc35-d211-4f9e-8a65-15dd4a56d9cf None admin] RBAC: Adding query filter params (role.id=30c027fab0344b47b1ba1ef6c6cbd818, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=7542e1fd160e4b2c9d25accf19aad19f) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:05.131559 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828adc35-d211-4f9e-8a65-15dd4a56d9cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:05.133724 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828adc35-d211-4f9e-8a65-15dd4a56d9cf None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:05.139681 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 549/1105] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:32:04 2022] GET /identity/v3/role_assignments?user.id=7542e1fd160e4b2c9d25accf19aad19f&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=30c027fab0344b47b1ba1ef6c6cbd818 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:32:05.204038 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:05.208030 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0905e017-ae06-4730-8363-67f644774bd0 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:05.248495 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea738e9-dcc7-49ab-a4e3-c415924fcb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:05.249204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea738e9-dcc7-49ab-a4e3-c415924fcb1f None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:05.249486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea738e9-dcc7-49ab-a4e3-c415924fcb1f None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:05.249760 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea738e9-dcc7-49ab-a4e3-c415924fcb1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:05.250614 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea738e9-dcc7-49ab-a4e3-c415924fcb1f None admin] RBAC: Adding query filter params (role.id=6505bbb129404436966ee59aa7a01d8e, scope.project.id=c0e4b22c01f241e9898771fd17fdd43d, user.id=640978855c9d4b14afe7d112957b80fb) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:05.250927 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea738e9-dcc7-49ab-a4e3-c415924fcb1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:05.252399 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea738e9-dcc7-49ab-a4e3-c415924fcb1f None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:05.259945 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 557/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:32:05 2022] GET /identity/v3/role_assignments?user.id=640978855c9d4b14afe7d112957b80fb&scope.project.id=c0e4b22c01f241e9898771fd17fdd43d&role.id=6505bbb129404436966ee59aa7a01d8e => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:32:07.526509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ef348-5322-4e30-8dd9-45c880181a6d None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:07.527187 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ef348-5322-4e30-8dd9-45c880181a6d None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:07.527187 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ef348-5322-4e30-8dd9-45c880181a6d None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:07.528434 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 550/1107] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:32:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:32:07.533736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900ae17-f487-4575-a278-09f2f49bd3be None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:07.534147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900ae17-f487-4575-a278-09f2f49bd3be None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:07.534328 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900ae17-f487-4575-a278-09f2f49bd3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:07.551071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-7900ae17-f487-4575-a278-09f2f49bd3be None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:32:07.552252 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-7900ae17-f487-4575-a278-09f2f49bd3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:32:07.619998 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 558/1108] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:32:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:32:07.644655 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ef348-5322-4e30-8dd9-45c880181a6d None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:07.744162 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ef348-5322-4e30-8dd9-45c880181a6d None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:07.777247 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcef7b7-d9bf-4743-80de-eb138bcc3be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:07.777962 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcef7b7-d9bf-4743-80de-eb138bcc3be1 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:07.778240 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcef7b7-d9bf-4743-80de-eb138bcc3be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:07.778558 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcef7b7-d9bf-4743-80de-eb138bcc3be1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:07.779717 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcef7b7-d9bf-4743-80de-eb138bcc3be1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:07.781174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcef7b7-d9bf-4743-80de-eb138bcc3be1 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:07.782783 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 551/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:32:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:07.788773 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900ae17-f487-4575-a278-09f2f49bd3be None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:07.885193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:07.893694 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900ae17-f487-4575-a278-09f2f49bd3be None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:07.928492 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4c62b-6bff-47ba-879e-6e1d0a999cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:07.929701 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4c62b-6bff-47ba-879e-6e1d0a999cb1 None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:07.930144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4c62b-6bff-47ba-879e-6e1d0a999cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:07.930425 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4c62b-6bff-47ba-879e-6e1d0a999cb1 None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:07.931163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af4c62b-6bff-47ba-879e-6e1d0a999cb1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:07.932664 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af4c62b-6bff-47ba-879e-6e1d0a999cb1 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:08.107846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 559/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Fri May 27 19:32:07 2022] POST /identity/v3/users => generated 286 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:32:10.418173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b94ed-0a42-40fc-a085-99f93f6bc81b None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:10.418753 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b94ed-0a42-40fc-a085-99f93f6bc81b None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:10.418862 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b94ed-0a42-40fc-a085-99f93f6bc81b None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:10.420170 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 552/1111] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:32:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:32:10.425513 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a208bb1-5896-4ef6-ab30-edf7d06613cd None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:10.425968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a208bb1-5896-4ef6-ab30-edf7d06613cd None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:10.426079 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a208bb1-5896-4ef6-ab30-edf7d06613cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:10.444109 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-7a208bb1-5896-4ef6-ab30-edf7d06613cd None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:32:10.445321 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-7a208bb1-5896-4ef6-ab30-edf7d06613cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:32:10.512746 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 560/1112] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:32:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:32:10.536855 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b94ed-0a42-40fc-a085-99f93f6bc81b None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:10.552134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:10.639697 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b94ed-0a42-40fc-a085-99f93f6bc81b None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:10.673338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b3bcb-63ae-453d-ad8f-5681d1d0c743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:10.674133 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b3bcb-63ae-453d-ad8f-5681d1d0c743 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:10.674418 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b3bcb-63ae-453d-ad8f-5681d1d0c743 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:10.674694 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5b3bcb-63ae-453d-ad8f-5681d1d0c743 None admin] PATH_INFO: `/v3/roles/eda4ac05bc364224a7f319e7ded54073` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:10.676166 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b3bcb-63ae-453d-ad8f-5681d1d0c743 None admin] RBAC: Authorizing `identity:get_role(role_id=eda4ac05bc364224a7f319e7ded54073)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:10.677575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5b3bcb-63ae-453d-ad8f-5681d1d0c743 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:10.679706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 553/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:32:10 2022] GET /identity/v3/roles/eda4ac05bc364224a7f319e7ded54073 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:10.685674 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a208bb1-5896-4ef6-ab30-edf7d06613cd None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:10.762364 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:10.782727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a208bb1-5896-4ef6-ab30-edf7d06613cd None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:10.817236 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f667aab-862f-4bd2-9acb-4fcea105487a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:10.818140 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f667aab-862f-4bd2-9acb-4fcea105487a None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:10.819033 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f667aab-862f-4bd2-9acb-4fcea105487a None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:10.819312 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f667aab-862f-4bd2-9acb-4fcea105487a None admin] PATH_INFO: `/v3/users/4603b654faaa45ee98de8d15cc699c7c` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:10.833416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f667aab-862f-4bd2-9acb-4fcea105487a None admin] RBAC: Authorizing `identity:get_user(user_id=4603b654faaa45ee98de8d15cc699c7c)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:10.835129 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f667aab-862f-4bd2-9acb-4fcea105487a None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:10.836824 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 561/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:32:10 2022] GET /identity/v3/users/4603b654faaa45ee98de8d15cc699c7c => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:10.842853 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b3bcb-63ae-453d-ad8f-5681d1d0c743 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:10.929034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:10.949694 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5b3bcb-63ae-453d-ad8f-5681d1d0c743 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:10.983039 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ade19a-5320-43ac-9b10-eb06d63acc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:10.983689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ade19a-5320-43ac-9b10-eb06d63acc2b None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:10.983974 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ade19a-5320-43ac-9b10-eb06d63acc2b None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:10.984252 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ade19a-5320-43ac-9b10-eb06d63acc2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:11.049797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ade19a-5320-43ac-9b10-eb06d63acc2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:11.051235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ade19a-5320-43ac-9b10-eb06d63acc2b None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:11.128664 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:11.149157 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 554/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:32:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 19:32:11.155287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f667aab-862f-4bd2-9acb-4fcea105487a None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:11.255339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f667aab-862f-4bd2-9acb-4fcea105487a None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:11.289994 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a84fe5-6483-40b9-a49d-22ea694c4964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:11.290855 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a84fe5-6483-40b9-a49d-22ea694c4964 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:11.291135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a84fe5-6483-40b9-a49d-22ea694c4964 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:11.291408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a84fe5-6483-40b9-a49d-22ea694c4964 None admin] PATH_INFO: `/v3/projects/c0e4b22c01f241e9898771fd17fdd43d` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:11.292589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a84fe5-6483-40b9-a49d-22ea694c4964 None admin] RBAC: Authorizing `identity:get_project(project_id=c0e4b22c01f241e9898771fd17fdd43d)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:11.294117 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a84fe5-6483-40b9-a49d-22ea694c4964 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:11.295821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 562/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:32:11 2022] GET /identity/v3/projects/c0e4b22c01f241e9898771fd17fdd43d => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:11.301980 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ade19a-5320-43ac-9b10-eb06d63acc2b None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:11.403181 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ade19a-5320-43ac-9b10-eb06d63acc2b None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:11.436926 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1126e9-f618-4872-9210-5550e7baba4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:11.437596 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1126e9-f618-4872-9210-5550e7baba4b None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:11.437876 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1126e9-f618-4872-9210-5550e7baba4b None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:11.438148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1126e9-f618-4872-9210-5550e7baba4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:11.438972 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1126e9-f618-4872-9210-5550e7baba4b None admin] RBAC: Adding query filter params (role.id=eda4ac05bc364224a7f319e7ded54073, scope.project.id=c0e4b22c01f241e9898771fd17fdd43d, user.id=4603b654faaa45ee98de8d15cc699c7c) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:11.439279 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1126e9-f618-4872-9210-5550e7baba4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:11.440702 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1126e9-f618-4872-9210-5550e7baba4b None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:11.446821 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 555/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:32:11 2022] GET /identity/v3/role_assignments?user.id=4603b654faaa45ee98de8d15cc699c7c&scope.project.id=c0e4b22c01f241e9898771fd17fdd43d&role.id=eda4ac05bc364224a7f319e7ded54073 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:13.635040 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8b8c5-7454-4def-aace-d852c16d584e None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:13.635561 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8b8c5-7454-4def-aace-d852c16d584e None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:13.635561 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f8b8c5-7454-4def-aace-d852c16d584e None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:13.636966 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 563/1118] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:32:13.642292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8d5ce-7a4d-4f7e-ab77-533e34f0ae26 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:13.642685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8d5ce-7a4d-4f7e-ab77-533e34f0ae26 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:13.642847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8d5ce-7a4d-4f7e-ab77-533e34f0ae26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:13.659562 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-bbd8d5ce-7a4d-4f7e-ab77-533e34f0ae26 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:32:13.660722 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-bbd8d5ce-7a4d-4f7e-ab77-533e34f0ae26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:32:13.700261 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:13.729647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 556/1119] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:32:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:32:13.751427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8b8c5-7454-4def-aace-d852c16d584e None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:13.755204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:13.854680 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f8b8c5-7454-4def-aace-d852c16d584e None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:13.887688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d249a146-b9fd-4c7b-85f3-62c704434f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:13.888534 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249a146-b9fd-4c7b-85f3-62c704434f54 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:13.888813 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249a146-b9fd-4c7b-85f3-62c704434f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:13.889117 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d249a146-b9fd-4c7b-85f3-62c704434f54 None admin] PATH_INFO: `/v3/roles/eda4ac05bc364224a7f319e7ded54073` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:13.890563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d249a146-b9fd-4c7b-85f3-62c704434f54 None admin] RBAC: Authorizing `identity:get_role(role_id=eda4ac05bc364224a7f319e7ded54073)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:13.891924 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d249a146-b9fd-4c7b-85f3-62c704434f54 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:13.893312 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 564/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:32:13 2022] GET /identity/v3/roles/eda4ac05bc364224a7f319e7ded54073 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:13.901008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8d5ce-7a4d-4f7e-ab77-533e34f0ae26 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:14.002262 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8d5ce-7a4d-4f7e-ab77-533e34f0ae26 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:14.039140 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc631a7e-2f6e-40c2-beac-c1eedc6d8926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:14.040011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc631a7e-2f6e-40c2-beac-c1eedc6d8926 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:14.040296 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc631a7e-2f6e-40c2-beac-c1eedc6d8926 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:14.040576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc631a7e-2f6e-40c2-beac-c1eedc6d8926 None admin] PATH_INFO: `/v3/users/4603b654faaa45ee98de8d15cc699c7c` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:14.041779 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc631a7e-2f6e-40c2-beac-c1eedc6d8926 None admin] RBAC: Authorizing `identity:get_user(user_id=4603b654faaa45ee98de8d15cc699c7c)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:14.043310 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc631a7e-2f6e-40c2-beac-c1eedc6d8926 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:14.045142 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 557/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:32:13 2022] GET /identity/v3/users/4603b654faaa45ee98de8d15cc699c7c => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:14.050957 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d249a146-b9fd-4c7b-85f3-62c704434f54 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:14.120547 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:14.150844 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d249a146-b9fd-4c7b-85f3-62c704434f54 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:14.183398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8495bf61-9eeb-41a5-8748-3a8126817463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:14.184245 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495bf61-9eeb-41a5-8748-3a8126817463 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:14.184526 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495bf61-9eeb-41a5-8748-3a8126817463 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:14.184800 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8495bf61-9eeb-41a5-8748-3a8126817463 None admin] PATH_INFO: `/v3/projects/c0e4b22c01f241e9898771fd17fdd43d` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:14.186006 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8495bf61-9eeb-41a5-8748-3a8126817463 None admin] RBAC: Authorizing `identity:get_project(project_id=c0e4b22c01f241e9898771fd17fdd43d)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:14.187439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8495bf61-9eeb-41a5-8748-3a8126817463 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:14.189157 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 565/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:32:14 2022] GET /identity/v3/projects/c0e4b22c01f241e9898771fd17fdd43d => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:14.195851 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc631a7e-2f6e-40c2-beac-c1eedc6d8926 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:14.238548 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:14.299563 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc631a7e-2f6e-40c2-beac-c1eedc6d8926 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:14.333924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea74c47-4c20-414f-86ca-bad768bf6405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:14.334839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea74c47-4c20-414f-86ca-bad768bf6405 None admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:14.335120 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea74c47-4c20-414f-86ca-bad768bf6405 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:14.335395 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea74c47-4c20-414f-86ca-bad768bf6405 None admin] PATH_INFO: `/v3/projects/c0e4b22c01f241e9898771fd17fdd43d/users/4603b654faaa45ee98de8d15cc699c7c/roles/eda4ac05bc364224a7f319e7ded54073` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:14.337595 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea74c47-4c20-414f-86ca-bad768bf6405 None admin] RBAC: Authorizing `identity:create_grant(project_id=c0e4b22c01f241e9898771fd17fdd43d, user_id=4603b654faaa45ee98de8d15cc699c7c, role_id=eda4ac05bc364224a7f319e7ded54073)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:14.339019 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea74c47-4c20-414f-86ca-bad768bf6405 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:14.436114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 558/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:32:14 2022] PUT /identity/v3/projects/c0e4b22c01f241e9898771fd17fdd43d/users/4603b654faaa45ee98de8d15cc699c7c/roles/eda4ac05bc364224a7f319e7ded54073 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:32:16.625611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5121ef2-e1e3-45fc-8b42-aac13ba40c45 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:16.626165 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5121ef2-e1e3-45fc-8b42-aac13ba40c45 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:16.626165 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5121ef2-e1e3-45fc-8b42-aac13ba40c45 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:16.627531 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 566/1124] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:32:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:32:16.632938 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0125db-0493-46b7-a6f5-9f0a8423b2fd None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:16.633343 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0125db-0493-46b7-a6f5-9f0a8423b2fd None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:16.633524 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0125db-0493-46b7-a6f5-9f0a8423b2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:16.650771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-9d0125db-0493-46b7-a6f5-9f0a8423b2fd None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:32:16.651963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-9d0125db-0493-46b7-a6f5-9f0a8423b2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:32:16.717991 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 559/1125] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:32:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:32:16.741922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5121ef2-e1e3-45fc-8b42-aac13ba40c45 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:16.844380 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5121ef2-e1e3-45fc-8b42-aac13ba40c45 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:16.878586 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01a63c-dea7-40ba-a2ad-90dd5c4db51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:16.879450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01a63c-dea7-40ba-a2ad-90dd5c4db51e None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:16.880380 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01a63c-dea7-40ba-a2ad-90dd5c4db51e None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:16.880841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01a63c-dea7-40ba-a2ad-90dd5c4db51e None admin] PATH_INFO: `/v3/roles/eda4ac05bc364224a7f319e7ded54073` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:16.882354 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01a63c-dea7-40ba-a2ad-90dd5c4db51e None admin] RBAC: Authorizing `identity:get_role(role_id=eda4ac05bc364224a7f319e7ded54073)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:16.883776 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01a63c-dea7-40ba-a2ad-90dd5c4db51e None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:16.885174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 567/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:32:16 2022] GET /identity/v3/roles/eda4ac05bc364224a7f319e7ded54073 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:16.891129 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0125db-0493-46b7-a6f5-9f0a8423b2fd None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:16.921129 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:16.991718 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0125db-0493-46b7-a6f5-9f0a8423b2fd None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:17.027388 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3916f-2967-40cc-80aa-40f5e4f1de6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:17.028444 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3916f-2967-40cc-80aa-40f5e4f1de6a None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:17.028727 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3916f-2967-40cc-80aa-40f5e4f1de6a None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:17.029025 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3916f-2967-40cc-80aa-40f5e4f1de6a None admin] PATH_INFO: `/v3/users/4603b654faaa45ee98de8d15cc699c7c` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:17.030271 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3916f-2967-40cc-80aa-40f5e4f1de6a None admin] RBAC: Authorizing `identity:get_user(user_id=4603b654faaa45ee98de8d15cc699c7c)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:17.031667 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3916f-2967-40cc-80aa-40f5e4f1de6a None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:17.033339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 560/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:32:16 2022] GET /identity/v3/users/4603b654faaa45ee98de8d15cc699c7c => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:17.040221 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01a63c-dea7-40ba-a2ad-90dd5c4db51e None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:17.053428 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:17.142731 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01a63c-dea7-40ba-a2ad-90dd5c4db51e None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:17.177539 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4594206-1cf9-4a69-a41c-7a38fbc7b67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:17.178223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4594206-1cf9-4a69-a41c-7a38fbc7b67e None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:17.178509 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4594206-1cf9-4a69-a41c-7a38fbc7b67e None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:17.178786 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4594206-1cf9-4a69-a41c-7a38fbc7b67e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:17.249099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4594206-1cf9-4a69-a41c-7a38fbc7b67e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:17.250604 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4594206-1cf9-4a69-a41c-7a38fbc7b67e None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:17.260979 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:17.355912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 568/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:32:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 19:32:17.361927 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3916f-2967-40cc-80aa-40f5e4f1de6a None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:17.461988 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:17.464899 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3916f-2967-40cc-80aa-40f5e4f1de6a None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:17.500031 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68d65c-71ef-462c-966b-8b202fb58999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:17.500891 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68d65c-71ef-462c-966b-8b202fb58999 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:17.501200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68d65c-71ef-462c-966b-8b202fb58999 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:17.501475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68d65c-71ef-462c-966b-8b202fb58999 None admin] PATH_INFO: `/v3/projects/c0e4b22c01f241e9898771fd17fdd43d` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:17.502671 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68d65c-71ef-462c-966b-8b202fb58999 None admin] RBAC: Authorizing `identity:get_project(project_id=c0e4b22c01f241e9898771fd17fdd43d)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:17.504094 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68d65c-71ef-462c-966b-8b202fb58999 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:17.506511 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 561/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:32:17 2022] GET /identity/v3/projects/c0e4b22c01f241e9898771fd17fdd43d => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:17.515225 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4594206-1cf9-4a69-a41c-7a38fbc7b67e None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:17.629104 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4594206-1cf9-4a69-a41c-7a38fbc7b67e None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:17.639443 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:17.665394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d49f0-cac7-4162-98d8-782b6e03a022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:17.666052 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d49f0-cac7-4162-98d8-782b6e03a022 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:17.666339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d49f0-cac7-4162-98d8-782b6e03a022 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:17.666610 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d49f0-cac7-4162-98d8-782b6e03a022 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:17.667451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d49f0-cac7-4162-98d8-782b6e03a022 None admin] RBAC: Adding query filter params (role.id=eda4ac05bc364224a7f319e7ded54073, scope.project.id=c0e4b22c01f241e9898771fd17fdd43d, user.id=4603b654faaa45ee98de8d15cc699c7c) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:17.667766 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d49f0-cac7-4162-98d8-782b6e03a022 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:17.669221 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d49f0-cac7-4162-98d8-782b6e03a022 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:17.675172 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 569/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:32:17 2022] GET /identity/v3/role_assignments?user.id=4603b654faaa45ee98de8d15cc699c7c&scope.project.id=c0e4b22c01f241e9898771fd17fdd43d&role.id=eda4ac05bc364224a7f319e7ded54073 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:19.967524 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a40642-000f-4130-842e-006b389af894 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:19.968078 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a40642-000f-4130-842e-006b389af894 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:19.968175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a40642-000f-4130-842e-006b389af894 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:19.969588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 562/1131] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:32:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:32:19.975569 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde88f4e-534c-4e6b-a93d-834d3fa06d48 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:19.975993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde88f4e-534c-4e6b-a93d-834d3fa06d48 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:19.976103 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde88f4e-534c-4e6b-a93d-834d3fa06d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:19.993126 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-cde88f4e-534c-4e6b-a93d-834d3fa06d48 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:32:19.994268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-cde88f4e-534c-4e6b-a93d-834d3fa06d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:32:20.059720 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 570/1132] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:32:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:32:20.084455 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a40642-000f-4130-842e-006b389af894 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:20.186378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a40642-000f-4130-842e-006b389af894 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:20.221758 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6455fd0-636d-4724-96a1-955eee0ad6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:20.222503 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6455fd0-636d-4724-96a1-955eee0ad6fe None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:20.222784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6455fd0-636d-4724-96a1-955eee0ad6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:20.223063 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6455fd0-636d-4724-96a1-955eee0ad6fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:20.225045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6455fd0-636d-4724-96a1-955eee0ad6fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:20.226586 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6455fd0-636d-4724-96a1-955eee0ad6fe None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:20.228334 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 563/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:32:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:20.234535 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde88f4e-534c-4e6b-a93d-834d3fa06d48 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:20.335150 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde88f4e-534c-4e6b-a93d-834d3fa06d48 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:20.336654 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:20.370281 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6238390f-92c2-4e21-a8d6-d9ba2560420a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:20.370928 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238390f-92c2-4e21-a8d6-d9ba2560420a None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:20.371208 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238390f-92c2-4e21-a8d6-d9ba2560420a None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:20.371483 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6238390f-92c2-4e21-a8d6-d9ba2560420a None admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:20.372179 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238390f-92c2-4e21-a8d6-d9ba2560420a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:20.373660 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6238390f-92c2-4e21-a8d6-d9ba2560420a None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:20.529661 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 571/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Fri May 27 19:32:20 2022] POST /identity/v3/projects => generated 311 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:32:22.787077 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3383d2-a6e0-494f-bc36-0f2447aa491d None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:22.787616 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3383d2-a6e0-494f-bc36-0f2447aa491d None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:22.787764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3383d2-a6e0-494f-bc36-0f2447aa491d None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:22.789198 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 564/1135] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:32:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:32:22.794841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528c911-f232-4db4-bc90-d97b8f9a285d None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:22.795289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528c911-f232-4db4-bc90-d97b8f9a285d None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:22.795400 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528c911-f232-4db4-bc90-d97b8f9a285d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:22.812866 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-7528c911-f232-4db4-bc90-d97b8f9a285d None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:32:22.814078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-7528c911-f232-4db4-bc90-d97b8f9a285d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:32:22.881058 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 572/1136] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:32:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 19:32:22.906532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3383d2-a6e0-494f-bc36-0f2447aa491d None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:22.941591 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:23.006208 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3383d2-a6e0-494f-bc36-0f2447aa491d None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:23.039274 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7bc39-9b0c-4b16-bba9-9189fb19724b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:23.039983 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7bc39-9b0c-4b16-bba9-9189fb19724b None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:23.040261 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7bc39-9b0c-4b16-bba9-9189fb19724b None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:23.040531 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7bc39-9b0c-4b16-bba9-9189fb19724b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:23.041800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7bc39-9b0c-4b16-bba9-9189fb19724b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:23.043219 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7bc39-9b0c-4b16-bba9-9189fb19724b None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:23.044811 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 565/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:32:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:23.050614 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528c911-f232-4db4-bc90-d97b8f9a285d None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:23.134302 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:23.150006 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528c911-f232-4db4-bc90-d97b8f9a285d None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:23.183436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952bd0e1-a0ea-42e8-a92c-2c217acf39d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:23.184106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bd0e1-a0ea-42e8-a92c-2c217acf39d5 None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:23.184391 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bd0e1-a0ea-42e8-a92c-2c217acf39d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:23.184668 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bd0e1-a0ea-42e8-a92c-2c217acf39d5 None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:23.185405 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952bd0e1-a0ea-42e8-a92c-2c217acf39d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:23.186855 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952bd0e1-a0ea-42e8-a92c-2c217acf39d5 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:23.274587 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 573/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Fri May 27 19:32:23 2022] POST /identity/v3/users => generated 286 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:32:25.646179 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fec9d-fcd3-4765-9500-e30cee874866 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:25.646805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fec9d-fcd3-4765-9500-e30cee874866 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:25.646805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fec9d-fcd3-4765-9500-e30cee874866 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:25.648131 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 566/1139] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:32:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:32:25.653710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa112d6-89a9-426d-9f09-4d13657af376 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:25.654147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa112d6-89a9-426d-9f09-4d13657af376 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:25.654275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa112d6-89a9-426d-9f09-4d13657af376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:25.672638 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-7fa112d6-89a9-426d-9f09-4d13657af376 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:32:25.673852 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-7fa112d6-89a9-426d-9f09-4d13657af376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:32:25.740630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 574/1140] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:32:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:32:25.763619 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fec9d-fcd3-4765-9500-e30cee874866 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:25.866973 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fec9d-fcd3-4765-9500-e30cee874866 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:25.869394 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:25.902055 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb11472-c0c2-4c8c-bd06-d50ed2be3e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:25.902836 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb11472-c0c2-4c8c-bd06-d50ed2be3e64 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:25.903119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb11472-c0c2-4c8c-bd06-d50ed2be3e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:25.903394 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb11472-c0c2-4c8c-bd06-d50ed2be3e64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:25.913500 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb11472-c0c2-4c8c-bd06-d50ed2be3e64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:32:25.921099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-0fb11472-c0c2-4c8c-bd06-d50ed2be3e64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:32:25.921099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:32:25.921099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:32:25.921099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:32:25.921099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:32:25.921099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:32:25.921099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:32:25.921099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:32:25.921099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:32:25.921099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:32:25.921099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:32:25.921099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:32:25.921099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:32:25.921099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:32:25.921099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:32:25.921099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:32:25.922305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:32:25.922305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:32:25.922305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:32:25.922305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:32:25.922305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:32:25.922305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:32:25.922305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:32:25.922305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:32:25.922305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:32:25.922305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:32:25.922305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:32:25.922305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:32:25.922305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:32:25.922305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:32:25.922305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:32:25.922305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:32:25.923410 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:32:25.923410 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:32:25.923410 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:32:25.923410 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:32:25.923410 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:32:25.923410 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:32:25.923410 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:32:25.923410 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:32:25.923410 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:32:25.923410 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:32:25.923410 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:32:25.923410 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:32:25.923410 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 567/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:32:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:32:25.928693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa112d6-89a9-426d-9f09-4d13657af376 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:25.995991 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:26.026495 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa112d6-89a9-426d-9f09-4d13657af376 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:26.060111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02387da4-2353-4cd5-ae55-24961ce656db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:26.060938 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02387da4-2353-4cd5-ae55-24961ce656db None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:26.061130 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02387da4-2353-4cd5-ae55-24961ce656db None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:26.061413 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02387da4-2353-4cd5-ae55-24961ce656db None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:26.062158 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02387da4-2353-4cd5-ae55-24961ce656db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:26.062476 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02387da4-2353-4cd5-ae55-24961ce656db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:26.063872 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02387da4-2353-4cd5-ae55-24961ce656db None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:26.073466 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 575/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:32:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:26.079479 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb11472-c0c2-4c8c-bd06-d50ed2be3e64 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:26.180669 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb11472-c0c2-4c8c-bd06-d50ed2be3e64 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:26.214101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95a0ef-5b58-4657-b97f-48004a3faad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:26.214980 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95a0ef-5b58-4657-b97f-48004a3faad8 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:26.215262 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95a0ef-5b58-4657-b97f-48004a3faad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:26.215541 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95a0ef-5b58-4657-b97f-48004a3faad8 None admin] PATH_INFO: `/v3/users/d31df9b9dd334e87952ce1c292164820` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:26.228567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95a0ef-5b58-4657-b97f-48004a3faad8 None admin] RBAC: Authorizing `identity:get_user(user_id=d31df9b9dd334e87952ce1c292164820)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:26.230037 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95a0ef-5b58-4657-b97f-48004a3faad8 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:26.231686 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 568/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:32:26 2022] GET /identity/v3/users/d31df9b9dd334e87952ce1c292164820 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:26.237717 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02387da4-2353-4cd5-ae55-24961ce656db None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:26.335655 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02387da4-2353-4cd5-ae55-24961ce656db None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:26.363814 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:26.369166 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea147fb-baad-48fc-addf-f1befcac9b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:26.369815 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea147fb-baad-48fc-addf-f1befcac9b12 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:26.370097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea147fb-baad-48fc-addf-f1befcac9b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:26.370372 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea147fb-baad-48fc-addf-f1befcac9b12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:26.436561 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea147fb-baad-48fc-addf-f1befcac9b12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:26.438052 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea147fb-baad-48fc-addf-f1befcac9b12 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:26.534688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 576/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:32:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 19:32:26.541406 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95a0ef-5b58-4657-b97f-48004a3faad8 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:26.565395 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:26.640249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95a0ef-5b58-4657-b97f-48004a3faad8 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:26.673903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f636ca-188b-4cc7-9561-dd09de1d4ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:26.674754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f636ca-188b-4cc7-9561-dd09de1d4ddd None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:26.675088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f636ca-188b-4cc7-9561-dd09de1d4ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:26.675359 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f636ca-188b-4cc7-9561-dd09de1d4ddd None admin] PATH_INFO: `/v3/projects/d6a9ff40aa1f4475a92077f885849a6f` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:26.677675 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f636ca-188b-4cc7-9561-dd09de1d4ddd None admin] RBAC: Authorizing `identity:get_project(project_id=d6a9ff40aa1f4475a92077f885849a6f)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:26.679438 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f636ca-188b-4cc7-9561-dd09de1d4ddd None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:26.681148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 569/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:32:26 2022] GET /identity/v3/projects/d6a9ff40aa1f4475a92077f885849a6f => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:26.687284 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea147fb-baad-48fc-addf-f1befcac9b12 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:26.717091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:26.784630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea147fb-baad-48fc-addf-f1befcac9b12 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:26.819059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e165d-994d-4c1e-877a-265d47c0d3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:26.819962 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e165d-994d-4c1e-877a-265d47c0d3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:26.820242 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e165d-994d-4c1e-877a-265d47c0d3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:26.820514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e165d-994d-4c1e-877a-265d47c0d3d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:26.821395 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e165d-994d-4c1e-877a-265d47c0d3d4 None admin] RBAC: Adding query filter params (role.id=6505bbb129404436966ee59aa7a01d8e, scope.project.id=d6a9ff40aa1f4475a92077f885849a6f, user.id=d31df9b9dd334e87952ce1c292164820) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:26.821667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e165d-994d-4c1e-877a-265d47c0d3d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:26.823087 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e165d-994d-4c1e-877a-265d47c0d3d4 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:26.828484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 577/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:32:26 2022] GET /identity/v3/role_assignments?user.id=d31df9b9dd334e87952ce1c292164820&scope.project.id=d6a9ff40aa1f4475a92077f885849a6f&role.id=6505bbb129404436966ee59aa7a01d8e => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:29.015442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de4fa7-2308-4883-b59b-d7899e2a8840 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:29.015985 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de4fa7-2308-4883-b59b-d7899e2a8840 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:29.015985 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de4fa7-2308-4883-b59b-d7899e2a8840 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:29.017381 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 570/1147] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:32:29.022878 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25855bf6-7b3e-4099-a3b7-87996ba9625a None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:29.023268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25855bf6-7b3e-4099-a3b7-87996ba9625a None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:29.023425 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25855bf6-7b3e-4099-a3b7-87996ba9625a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:29.041507 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-25855bf6-7b3e-4099-a3b7-87996ba9625a None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:32:29.042727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-25855bf6-7b3e-4099-a3b7-87996ba9625a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:32:29.109086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 578/1148] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:32:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:32:29.131968 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de4fa7-2308-4883-b59b-d7899e2a8840 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:29.224170 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:29.234275 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de4fa7-2308-4883-b59b-d7899e2a8840 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:29.270015 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f542dcb-4f34-4d3c-bf80-ba9211336325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:29.270731 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f542dcb-4f34-4d3c-bf80-ba9211336325 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:29.271017 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f542dcb-4f34-4d3c-bf80-ba9211336325 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:29.271317 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f542dcb-4f34-4d3c-bf80-ba9211336325 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:29.286911 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f542dcb-4f34-4d3c-bf80-ba9211336325 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:32:29.294647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-1f542dcb-4f34-4d3c-bf80-ba9211336325 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:32:29.294647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:32:29.294647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:32:29.294647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:32:29.294647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:32:29.294647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:32:29.294647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:32:29.294647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:32:29.294647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:32:29.294647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:32:29.294647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:32:29.294647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:32:29.294647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:32:29.294647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:32:29.294647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:32:29.294647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:32:29.295763 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:32:29.295763 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:32:29.295763 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:32:29.295763 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:32:29.295763 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:32:29.295763 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:32:29.295763 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:32:29.295763 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:32:29.295763 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:32:29.295763 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:32:29.295763 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:32:29.295763 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:32:29.295763 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:32:29.295763 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:32:29.295763 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:32:29.295763 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:32:29.296666 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:32:29.296666 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:32:29.296666 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:32:29.296666 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:32:29.296666 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:32:29.296666 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:32:29.296666 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:32:29.296666 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:32:29.296666 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:32:29.296666 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:32:29.296666 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:32:29.296666 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:32:29.296666 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 571/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:32:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:32:29.302337 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25855bf6-7b3e-4099-a3b7-87996ba9625a None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:29.329382 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:29.402459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25855bf6-7b3e-4099-a3b7-87996ba9625a None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:29.441472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ca6d7-09ee-4f2d-bfed-0a98cd41378c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:29.442234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ca6d7-09ee-4f2d-bfed-0a98cd41378c None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:29.442537 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ca6d7-09ee-4f2d-bfed-0a98cd41378c None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:29.442815 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ca6d7-09ee-4f2d-bfed-0a98cd41378c None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:29.443561 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ca6d7-09ee-4f2d-bfed-0a98cd41378c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:29.443868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ca6d7-09ee-4f2d-bfed-0a98cd41378c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:29.445535 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ca6d7-09ee-4f2d-bfed-0a98cd41378c None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:29.454572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 579/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:32:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:29.461278 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f542dcb-4f34-4d3c-bf80-ba9211336325 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:29.562916 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f542dcb-4f34-4d3c-bf80-ba9211336325 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:29.596623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85428e-9a40-4b91-9332-a3d9a95eb4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:29.598075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85428e-9a40-4b91-9332-a3d9a95eb4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:29.598445 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85428e-9a40-4b91-9332-a3d9a95eb4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:29.598722 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85428e-9a40-4b91-9332-a3d9a95eb4b8 None admin] PATH_INFO: `/v3/users/d31df9b9dd334e87952ce1c292164820` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:29.599923 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85428e-9a40-4b91-9332-a3d9a95eb4b8 None admin] RBAC: Authorizing `identity:get_user(user_id=d31df9b9dd334e87952ce1c292164820)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:29.601405 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85428e-9a40-4b91-9332-a3d9a95eb4b8 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:29.603548 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 572/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:32:29 2022] GET /identity/v3/users/d31df9b9dd334e87952ce1c292164820 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:29.609434 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ca6d7-09ee-4f2d-bfed-0a98cd41378c None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:29.696745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:29.708364 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ca6d7-09ee-4f2d-bfed-0a98cd41378c None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:29.744781 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb75093-ece5-48ed-8ff4-a5a072377ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:29.745667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb75093-ece5-48ed-8ff4-a5a072377ccb None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:29.745942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb75093-ece5-48ed-8ff4-a5a072377ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:29.746215 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb75093-ece5-48ed-8ff4-a5a072377ccb None admin] PATH_INFO: `/v3/projects/d6a9ff40aa1f4475a92077f885849a6f` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:29.747368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb75093-ece5-48ed-8ff4-a5a072377ccb None admin] RBAC: Authorizing `identity:get_project(project_id=d6a9ff40aa1f4475a92077f885849a6f)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:29.748805 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb75093-ece5-48ed-8ff4-a5a072377ccb None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:29.750526 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 580/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:32:29 2022] GET /identity/v3/projects/d6a9ff40aa1f4475a92077f885849a6f => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:29.756688 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85428e-9a40-4b91-9332-a3d9a95eb4b8 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:29.777310 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:29.859394 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85428e-9a40-4b91-9332-a3d9a95eb4b8 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:29.892633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dd04b5-c167-47a7-941e-2510c156529f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:29.893577 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd04b5-c167-47a7-941e-2510c156529f None admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:29.894035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd04b5-c167-47a7-941e-2510c156529f None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:29.894225 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dd04b5-c167-47a7-941e-2510c156529f None admin] PATH_INFO: `/v3/projects/d6a9ff40aa1f4475a92077f885849a6f/users/d31df9b9dd334e87952ce1c292164820/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:29.896133 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd04b5-c167-47a7-941e-2510c156529f None admin] RBAC: Authorizing `identity:create_grant(project_id=d6a9ff40aa1f4475a92077f885849a6f, user_id=d31df9b9dd334e87952ce1c292164820, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:29.897628 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dd04b5-c167-47a7-941e-2510c156529f None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:29.988028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 573/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:32:29 2022] PUT /identity/v3/projects/d6a9ff40aa1f4475a92077f885849a6f/users/d31df9b9dd334e87952ce1c292164820/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:32:32.185649 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210d150-e64f-4de7-bc92-0ae8f20888bb None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:32.186245 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210d150-e64f-4de7-bc92-0ae8f20888bb None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:32.186245 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9210d150-e64f-4de7-bc92-0ae8f20888bb None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:32.187580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 581/1154] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:32:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:32:32.193999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69776eb-9252-428b-9702-529d63f73569 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:32.194534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69776eb-9252-428b-9702-529d63f73569 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:32.194534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69776eb-9252-428b-9702-529d63f73569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:32.211204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-d69776eb-9252-428b-9702-529d63f73569 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:32:32.212357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d69776eb-9252-428b-9702-529d63f73569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:32:32.281349 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 574/1155] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:32:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:32:32.305865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210d150-e64f-4de7-bc92-0ae8f20888bb None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:32.406350 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9210d150-e64f-4de7-bc92-0ae8f20888bb None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:32.439426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0c12f-abf5-4aaa-9258-77ef0c50a924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:32.440164 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0c12f-abf5-4aaa-9258-77ef0c50a924 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:32.440442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0c12f-abf5-4aaa-9258-77ef0c50a924 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:32.440713 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f0c12f-abf5-4aaa-9258-77ef0c50a924 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:32.450843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f0c12f-abf5-4aaa-9258-77ef0c50a924 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:32:32.459071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-65f0c12f-abf5-4aaa-9258-77ef0c50a924 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:32:32.459071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:32:32.459071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:32:32.459071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:32:32.459071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:32:32.459071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:32:32.459071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:32:32.459071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:32:32.459071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:32:32.459071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:32:32.459071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:32:32.459071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:32:32.459071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:32:32.459071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:32:32.459071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:32:32.459071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:32:32.460289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:32:32.460289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:32:32.460289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:32:32.460289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:32:32.460289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:32:32.460289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:32:32.460289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:32:32.460289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:32:32.460289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:32:32.460289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:32:32.460289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:32:32.460289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:32:32.460289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:32:32.460289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:32:32.460289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:32:32.460289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:32:32.461414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:32:32.461414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:32:32.461414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:32:32.461414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:32:32.461414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:32:32.461414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:32:32.461414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:32:32.461414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:32:32.461414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:32:32.461414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:32:32.461414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:32:32.461414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:32:32.461414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 582/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:32:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:32:32.466310 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69776eb-9252-428b-9702-529d63f73569 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:32.477289 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:32.567961 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69776eb-9252-428b-9702-529d63f73569 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:32.601373 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cc461-6d97-466e-87a6-18f46c307bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:32.602042 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cc461-6d97-466e-87a6-18f46c307bdd None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:32.602328 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cc461-6d97-466e-87a6-18f46c307bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:32.602608 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cc461-6d97-466e-87a6-18f46c307bdd None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:32.603358 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404cc461-6d97-466e-87a6-18f46c307bdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:32.603669 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404cc461-6d97-466e-87a6-18f46c307bdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:32.605090 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404cc461-6d97-466e-87a6-18f46c307bdd None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:32.614120 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 575/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:32:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:32.619955 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0c12f-abf5-4aaa-9258-77ef0c50a924 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:32.638779 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:32.719461 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f0c12f-abf5-4aaa-9258-77ef0c50a924 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:32.754149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929c82e-099f-4fd0-9fe2-2429b2a40e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:32.755028 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929c82e-099f-4fd0-9fe2-2429b2a40e52 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:32.755315 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929c82e-099f-4fd0-9fe2-2429b2a40e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:32.755595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4929c82e-099f-4fd0-9fe2-2429b2a40e52 None admin] PATH_INFO: `/v3/users/d31df9b9dd334e87952ce1c292164820` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:32.756821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929c82e-099f-4fd0-9fe2-2429b2a40e52 None admin] RBAC: Authorizing `identity:get_user(user_id=d31df9b9dd334e87952ce1c292164820)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:32.758309 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4929c82e-099f-4fd0-9fe2-2429b2a40e52 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:32.760017 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 583/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:32:32 2022] GET /identity/v3/users/d31df9b9dd334e87952ce1c292164820 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:32.766005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cc461-6d97-466e-87a6-18f46c307bdd None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:32.836249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:32.868000 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cc461-6d97-466e-87a6-18f46c307bdd None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:32.904684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e19b1-16f1-4892-b0ef-e06b89d3ae71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:32.905373 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e19b1-16f1-4892-b0ef-e06b89d3ae71 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:32.905657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e19b1-16f1-4892-b0ef-e06b89d3ae71 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:32.905934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e19b1-16f1-4892-b0ef-e06b89d3ae71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:32.972671 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e19b1-16f1-4892-b0ef-e06b89d3ae71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:32.974078 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e19b1-16f1-4892-b0ef-e06b89d3ae71 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:33.039841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:33.071665 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 576/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:32:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 19:32:33.078000 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929c82e-099f-4fd0-9fe2-2429b2a40e52 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:33.166367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:33.182445 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4929c82e-099f-4fd0-9fe2-2429b2a40e52 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:33.218000 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d1f2e-3226-4480-a685-a64b0f5903d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:33.218990 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d1f2e-3226-4480-a685-a64b0f5903d6 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:33.219324 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d1f2e-3226-4480-a685-a64b0f5903d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:33.219613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d1f2e-3226-4480-a685-a64b0f5903d6 None admin] PATH_INFO: `/v3/projects/d6a9ff40aa1f4475a92077f885849a6f` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:33.220810 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d1f2e-3226-4480-a685-a64b0f5903d6 None admin] RBAC: Authorizing `identity:get_project(project_id=d6a9ff40aa1f4475a92077f885849a6f)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:33.222796 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607d1f2e-3226-4480-a685-a64b0f5903d6 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:33.224466 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 584/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:32:33 2022] GET /identity/v3/projects/d6a9ff40aa1f4475a92077f885849a6f => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:33.233190 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e19b1-16f1-4892-b0ef-e06b89d3ae71 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:33.331774 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e19b1-16f1-4892-b0ef-e06b89d3ae71 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:33.364719 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7580cd9-f477-4a74-82b5-6e04b760ddfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:33.365397 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7580cd9-f477-4a74-82b5-6e04b760ddfa None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:33.365678 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7580cd9-f477-4a74-82b5-6e04b760ddfa None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:33.365951 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7580cd9-f477-4a74-82b5-6e04b760ddfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:33.366766 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7580cd9-f477-4a74-82b5-6e04b760ddfa None admin] RBAC: Adding query filter params (role.id=6505bbb129404436966ee59aa7a01d8e, scope.project.id=d6a9ff40aa1f4475a92077f885849a6f, user.id=d31df9b9dd334e87952ce1c292164820) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:33.367072 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7580cd9-f477-4a74-82b5-6e04b760ddfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:33.368485 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7580cd9-f477-4a74-82b5-6e04b760ddfa None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:33.374242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 577/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:32:33 2022] GET /identity/v3/role_assignments?user.id=d31df9b9dd334e87952ce1c292164820&scope.project.id=d6a9ff40aa1f4475a92077f885849a6f&role.id=6505bbb129404436966ee59aa7a01d8e => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:35.636962 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682381f-866c-4467-bf72-ed05e6bd362e None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:35.637529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682381f-866c-4467-bf72-ed05e6bd362e None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:35.637529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682381f-866c-4467-bf72-ed05e6bd362e None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:35.638899 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 585/1162] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:32:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:32:35.644504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1d54f-f071-4fdf-af07-b61750e21e33 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:35.644944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1d54f-f071-4fdf-af07-b61750e21e33 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:35.645077 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1d54f-f071-4fdf-af07-b61750e21e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:35.661665 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-79f1d54f-f071-4fdf-af07-b61750e21e33 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:32:35.662839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-79f1d54f-f071-4fdf-af07-b61750e21e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:32:35.688227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:35.728294 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 578/1163] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:32:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:32:35.750522 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682381f-866c-4467-bf72-ed05e6bd362e None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:35.852033 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682381f-866c-4467-bf72-ed05e6bd362e None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:35.885482 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9e55c-567f-4070-a1cf-83874355b190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:35.886121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9e55c-567f-4070-a1cf-83874355b190 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:35.886398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9e55c-567f-4070-a1cf-83874355b190 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:35.886672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d9e55c-567f-4070-a1cf-83874355b190 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:35.907460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:35.953190 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d9e55c-567f-4070-a1cf-83874355b190 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:35.954641 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d9e55c-567f-4070-a1cf-83874355b190 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:36.050499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 586/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:32:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 19:32:36.056633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1d54f-f071-4fdf-af07-b61750e21e33 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:36.151982 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1d54f-f071-4fdf-af07-b61750e21e33 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:36.186854 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfd0276-da08-4fbe-8e3f-f82b5afd4433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:36.187652 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd0276-da08-4fbe-8e3f-f82b5afd4433 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:36.187936 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd0276-da08-4fbe-8e3f-f82b5afd4433 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:36.188211 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd0276-da08-4fbe-8e3f-f82b5afd4433 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:36.195967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd0276-da08-4fbe-8e3f-f82b5afd4433 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:36.197456 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfd0276-da08-4fbe-8e3f-f82b5afd4433 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:36.211852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-1dfd0276-da08-4fbe-8e3f-f82b5afd4433 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 27 19:32:36.211852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:32:36.211852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 27 19:32:36.211852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 27 19:32:36.211852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:32:36.211852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:32:36.211852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:32:36.211852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:32:36.211852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 27 19:32:36.211852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:32:36.211852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:32:36.211852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:32:36.211852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:32:36.211852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:32:36.211852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:32:36.211852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:32:36.211852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:32:36.213132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:32:36.213132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:32:36.213132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:32:36.213132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:32:36.213132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:32:36.213132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:32:36.213132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:32:36.213132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:32:36.213132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:32:36.213132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:32:36.213132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:32:36.213132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:32:36.213132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:32:36.213132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:32:36.213132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:32:36.213132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:32:36.214242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:32:36.214242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:32:36.214242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:32:36.214242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:32:36.214242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:32:36.214242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:32:36.214242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:32:36.214242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:32:36.214242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:32:36.214242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:32:36.214242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 27 19:32:36.214242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:32:36.214242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 27 19:32:36.214242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:32:36.214242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 579/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Fri May 27 19:32:36 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:32:36.219252 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9e55c-567f-4070-a1cf-83874355b190 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:36.274941 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:36.322312 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d9e55c-567f-4070-a1cf-83874355b190 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:36.355644 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4a3b3-657d-4c13-acb5-fcabbafb530f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:36.356316 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4a3b3-657d-4c13-acb5-fcabbafb530f None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:36.356605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4a3b3-657d-4c13-acb5-fcabbafb530f None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:36.356888 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4a3b3-657d-4c13-acb5-fcabbafb530f None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:36.357640 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4a3b3-657d-4c13-acb5-fcabbafb530f None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:36.357951 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4a3b3-657d-4c13-acb5-fcabbafb530f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:36.359341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4a3b3-657d-4c13-acb5-fcabbafb530f None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:36.367924 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 587/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Fri May 27 19:32:36 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:38.570565 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c3b2c-1306-45d2-8a8d-eb685cfab22b None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:38.571094 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c3b2c-1306-45d2-8a8d-eb685cfab22b None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:38.571094 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c3b2c-1306-45d2-8a8d-eb685cfab22b None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:38.572491 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 580/1167] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:32:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:32:38.577619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1ac3d-32b9-42f4-9e9d-59a0726dc1ed None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:38.578067 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1ac3d-32b9-42f4-9e9d-59a0726dc1ed None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:38.578193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb1ac3d-32b9-42f4-9e9d-59a0726dc1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:38.594945 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-fdb1ac3d-32b9-42f4-9e9d-59a0726dc1ed None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:32:38.596445 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-fdb1ac3d-32b9-42f4-9e9d-59a0726dc1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:32:38.663896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 588/1168] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:32:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:32:38.687128 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c3b2c-1306-45d2-8a8d-eb685cfab22b None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:38.713950 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:38.789171 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c3b2c-1306-45d2-8a8d-eb685cfab22b None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:38.823639 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7ff5dc-870e-425b-ad99-bc8295f1657f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:38.824300 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ff5dc-870e-425b-ad99-bc8295f1657f None admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:38.824589 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ff5dc-870e-425b-ad99-bc8295f1657f None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:38.824869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7ff5dc-870e-425b-ad99-bc8295f1657f None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:38.825646 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ff5dc-870e-425b-ad99-bc8295f1657f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:38.827020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7ff5dc-870e-425b-ad99-bc8295f1657f None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:38.895200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 581/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Fri May 27 19:32:38 2022] POST /identity/v3/domains => generated 267 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:32:41.134630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274dfed-d8b4-452e-9a91-48c8f0fbde6f None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:41.135197 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274dfed-d8b4-452e-9a91-48c8f0fbde6f None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:41.135197 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274dfed-d8b4-452e-9a91-48c8f0fbde6f None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:41.136548 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 589/1170] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:32:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:32:41.142910 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34859c2a-1fbd-4e95-8193-75219efa80f4 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:41.143448 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34859c2a-1fbd-4e95-8193-75219efa80f4 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:41.143448 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34859c2a-1fbd-4e95-8193-75219efa80f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:41.161741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-34859c2a-1fbd-4e95-8193-75219efa80f4 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:32:41.162906 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-34859c2a-1fbd-4e95-8193-75219efa80f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:32:41.229044 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 582/1171] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:32:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:32:41.253136 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274dfed-d8b4-452e-9a91-48c8f0fbde6f None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:41.300890 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:41.353885 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274dfed-d8b4-452e-9a91-48c8f0fbde6f None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:41.386546 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db4e29-9b65-4246-81aa-ae279b6c556a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:41.387341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db4e29-9b65-4246-81aa-ae279b6c556a None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:41.387704 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db4e29-9b65-4246-81aa-ae279b6c556a None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:41.387984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db4e29-9b65-4246-81aa-ae279b6c556a None admin] PATH_INFO: `/v3/domains/de029b6152ae4380a328655b0b2c36e0` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:41.399673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db4e29-9b65-4246-81aa-ae279b6c556a None admin] RBAC: Authorizing `identity:get_domain(domain_id=de029b6152ae4380a328655b0b2c36e0)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:41.401164 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db4e29-9b65-4246-81aa-ae279b6c556a None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:41.402834 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 590/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Fri May 27 19:32:41 2022] GET /identity/v3/domains/de029b6152ae4380a328655b0b2c36e0 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:41.408884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34859c2a-1fbd-4e95-8193-75219efa80f4 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:41.423597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:41.510189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34859c2a-1fbd-4e95-8193-75219efa80f4 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:41.545513 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31319efc-652d-40a2-9016-833f57312530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:41.546187 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31319efc-652d-40a2-9016-833f57312530 None admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:41.546499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31319efc-652d-40a2-9016-833f57312530 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:41.546775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31319efc-652d-40a2-9016-833f57312530 None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:41.547538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31319efc-652d-40a2-9016-833f57312530 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:41.549053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31319efc-652d-40a2-9016-833f57312530 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:41.679160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 583/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Fri May 27 19:32:41 2022] POST /identity/v3/projects => generated 361 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:32:43.930409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fda88-1d45-415e-9ce8-321a82d8394b None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:43.930921 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fda88-1d45-415e-9ce8-321a82d8394b None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:43.930921 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fda88-1d45-415e-9ce8-321a82d8394b None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:43.932336 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 591/1174] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:32:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:32:43.937883 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77012f22-f9e4-4163-8453-236ae396484f None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:43.938331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77012f22-f9e4-4163-8453-236ae396484f None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:43.938447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77012f22-f9e4-4163-8453-236ae396484f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:43.957001 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-77012f22-f9e4-4163-8453-236ae396484f None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:32:43.958173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-77012f22-f9e4-4163-8453-236ae396484f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:32:44.017723 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:44.025206 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 584/1175] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:32:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:32:44.051197 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3fda88-1d45-415e-9ce8-321a82d8394b None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:44.154235 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3fda88-1d45-415e-9ce8-321a82d8394b None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:44.162230 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:44.187593 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11576fab-718c-4740-af91-6e06b735eb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:44.188370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11576fab-718c-4740-af91-6e06b735eb62 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:44.188649 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11576fab-718c-4740-af91-6e06b735eb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:44.188983 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11576fab-718c-4740-af91-6e06b735eb62 None admin] PATH_INFO: `/v3/domains/de029b6152ae4380a328655b0b2c36e0` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:44.190163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11576fab-718c-4740-af91-6e06b735eb62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=de029b6152ae4380a328655b0b2c36e0)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:44.191862 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11576fab-718c-4740-af91-6e06b735eb62 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:44.193494 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 592/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Fri May 27 19:32:44 2022] GET /identity/v3/domains/de029b6152ae4380a328655b0b2c36e0 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:44.199277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77012f22-f9e4-4163-8453-236ae396484f None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:44.297407 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77012f22-f9e4-4163-8453-236ae396484f None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:44.332642 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55892b73-2d66-4461-829a-528fd2d2833d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:44.333353 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55892b73-2d66-4461-829a-528fd2d2833d None admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:44.333678 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55892b73-2d66-4461-829a-528fd2d2833d None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:44.334033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55892b73-2d66-4461-829a-528fd2d2833d None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:44.334804 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55892b73-2d66-4461-829a-528fd2d2833d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:44.336236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55892b73-2d66-4461-829a-528fd2d2833d None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:44.482426 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 585/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Fri May 27 19:32:44 2022] POST /identity/v3/users => generated 311 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:32:46.757441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483937e-4b00-4818-a76c-41362e0d92d0 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:46.758005 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483937e-4b00-4818-a76c-41362e0d92d0 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:46.758005 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3483937e-4b00-4818-a76c-41362e0d92d0 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:46.759368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 593/1178] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:32:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:32:46.764599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ac084-430a-47f2-996a-663d5cdef9f0 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:46.765110 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ac084-430a-47f2-996a-663d5cdef9f0 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:46.765356 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ac084-430a-47f2-996a-663d5cdef9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:46.782857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-8f3ac084-430a-47f2-996a-663d5cdef9f0 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:32:46.784033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-8f3ac084-430a-47f2-996a-663d5cdef9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:32:46.831421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:46.851242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 586/1179] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:32:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:32:46.875029 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483937e-4b00-4818-a76c-41362e0d92d0 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:46.974779 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3483937e-4b00-4818-a76c-41362e0d92d0 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:47.007701 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb88709-606e-4259-a313-a68372d584c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:47.008489 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb88709-606e-4259-a313-a68372d584c4 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:47.008772 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb88709-606e-4259-a313-a68372d584c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:47.009079 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb88709-606e-4259-a313-a68372d584c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:47.019607 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb88709-606e-4259-a313-a68372d584c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:32:47.027070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-beb88709-606e-4259-a313-a68372d584c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:32:47.027070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:32:47.027070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:32:47.027070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:32:47.027070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:32:47.027070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:32:47.027070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:32:47.027070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:32:47.027070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:32:47.027070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:32:47.027070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:32:47.027070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:32:47.027070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:32:47.027070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:32:47.027070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:32:47.027070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:32:47.028204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:32:47.028204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:32:47.028204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:32:47.028204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:32:47.028204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:32:47.028204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:32:47.028204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:32:47.028204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:32:47.028204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:32:47.028204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:32:47.028204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:32:47.028204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:32:47.028204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:32:47.028204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:32:47.028204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:32:47.028204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:32:47.033954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:32:47.033954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:32:47.033954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:32:47.033954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:32:47.033954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:32:47.033954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:32:47.033954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:32:47.033954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:32:47.033954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:32:47.033954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:32:47.033954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:32:47.033954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:32:47.033954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 594/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:32:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:32:47.035113 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ac084-430a-47f2-996a-663d5cdef9f0 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:47.134658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ac084-430a-47f2-996a-663d5cdef9f0 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:47.167462 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cccd98-7f87-4bb8-8100-995cbd6b2a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:47.168124 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cccd98-7f87-4bb8-8100-995cbd6b2a06 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:47.168407 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cccd98-7f87-4bb8-8100-995cbd6b2a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:47.168682 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cccd98-7f87-4bb8-8100-995cbd6b2a06 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:47.169458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cccd98-7f87-4bb8-8100-995cbd6b2a06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:47.169764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cccd98-7f87-4bb8-8100-995cbd6b2a06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:47.171847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cccd98-7f87-4bb8-8100-995cbd6b2a06 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:47.179744 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 587/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:32:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:47.185702 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb88709-606e-4259-a313-a68372d584c4 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:47.222114 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:47.289042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb88709-606e-4259-a313-a68372d584c4 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:47.323226 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04651248-1c69-403b-a918-70f24937753e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:47.324094 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04651248-1c69-403b-a918-70f24937753e None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:47.324373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04651248-1c69-403b-a918-70f24937753e None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:47.324644 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04651248-1c69-403b-a918-70f24937753e None admin] PATH_INFO: `/v3/users/14d65c39f9e344bea3244c453d38a8b9` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:47.337536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04651248-1c69-403b-a918-70f24937753e None admin] RBAC: Authorizing `identity:get_user(user_id=14d65c39f9e344bea3244c453d38a8b9)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:47.338974 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04651248-1c69-403b-a918-70f24937753e None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:47.340595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 595/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:32:47 2022] GET /identity/v3/users/14d65c39f9e344bea3244c453d38a8b9 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:47.347069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cccd98-7f87-4bb8-8100-995cbd6b2a06 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:47.392327 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:47.445732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cccd98-7f87-4bb8-8100-995cbd6b2a06 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:47.480032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa49af4-d940-4eec-8601-92b38f46cadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:47.480681 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa49af4-d940-4eec-8601-92b38f46cadc None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:47.480987 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa49af4-d940-4eec-8601-92b38f46cadc None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:47.481265 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa49af4-d940-4eec-8601-92b38f46cadc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:47.547896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa49af4-d940-4eec-8601-92b38f46cadc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:47.549298 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa49af4-d940-4eec-8601-92b38f46cadc None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:47.592925 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:47.647460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 588/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:32:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 19:32:47.653485 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04651248-1c69-403b-a918-70f24937753e None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:47.748165 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:47.757485 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04651248-1c69-403b-a918-70f24937753e None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:47.793824 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2766907-811c-4a8d-9e09-a313d2bdf4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:47.794669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2766907-811c-4a8d-9e09-a313d2bdf4ca None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:47.794949 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2766907-811c-4a8d-9e09-a313d2bdf4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:47.795223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2766907-811c-4a8d-9e09-a313d2bdf4ca None admin] PATH_INFO: `/v3/projects/6f9e46a6cd6b4682b3e3c29c1ac06cd1` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:47.797033 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2766907-811c-4a8d-9e09-a313d2bdf4ca None admin] RBAC: Authorizing `identity:get_project(project_id=6f9e46a6cd6b4682b3e3c29c1ac06cd1)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:47.798510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2766907-811c-4a8d-9e09-a313d2bdf4ca None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:47.800188 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 596/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:32:47 2022] GET /identity/v3/projects/6f9e46a6cd6b4682b3e3c29c1ac06cd1 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:47.806518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa49af4-d940-4eec-8601-92b38f46cadc None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:47.907088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa49af4-d940-4eec-8601-92b38f46cadc None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:47.943136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19909e-25d7-43f1-b7c4-2db7b60a6b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:47.943801 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19909e-25d7-43f1-b7c4-2db7b60a6b27 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:47.944111 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19909e-25d7-43f1-b7c4-2db7b60a6b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:47.944393 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19909e-25d7-43f1-b7c4-2db7b60a6b27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:47.945276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19909e-25d7-43f1-b7c4-2db7b60a6b27 None admin] RBAC: Adding query filter params (role.id=6505bbb129404436966ee59aa7a01d8e, scope.project.id=6f9e46a6cd6b4682b3e3c29c1ac06cd1, user.id=14d65c39f9e344bea3244c453d38a8b9) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:47.945656 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19909e-25d7-43f1-b7c4-2db7b60a6b27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:47.947023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19909e-25d7-43f1-b7c4-2db7b60a6b27 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:47.952805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 589/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:32:47 2022] GET /identity/v3/role_assignments?user.id=14d65c39f9e344bea3244c453d38a8b9&scope.project.id=6f9e46a6cd6b4682b3e3c29c1ac06cd1&role.id=6505bbb129404436966ee59aa7a01d8e => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:50.191906 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9558cd7-5afc-4076-beac-2419655ea77d None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:50.192449 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9558cd7-5afc-4076-beac-2419655ea77d None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:50.192449 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9558cd7-5afc-4076-beac-2419655ea77d None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:50.193852 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 597/1186] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:32:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:32:50.199369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2231f3-1144-40d5-adb5-427a440a607d None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:50.199808 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2231f3-1144-40d5-adb5-427a440a607d None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:50.199933 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2231f3-1144-40d5-adb5-427a440a607d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:50.216686 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-bf2231f3-1144-40d5-adb5-427a440a607d None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:32:50.217920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-bf2231f3-1144-40d5-adb5-427a440a607d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:32:50.220471 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:50.286035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 590/1187] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:32:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:32:50.309608 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9558cd7-5afc-4076-beac-2419655ea77d None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:50.409606 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9558cd7-5afc-4076-beac-2419655ea77d None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:50.443338 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a3179-e9b8-43ab-bf92-beab2a929120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:50.444044 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a3179-e9b8-43ab-bf92-beab2a929120 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:50.444327 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a3179-e9b8-43ab-bf92-beab2a929120 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:50.444603 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a3179-e9b8-43ab-bf92-beab2a929120 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:50.455738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a3179-e9b8-43ab-bf92-beab2a929120 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:32:50.463286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-564a3179-e9b8-43ab-bf92-beab2a929120 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:32:50.463286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:32:50.463286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:32:50.463286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:32:50.463286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:32:50.463286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:32:50.463286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:32:50.463286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:32:50.463286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:32:50.463286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:32:50.463286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:32:50.463286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:32:50.463286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:32:50.463286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:32:50.463286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:32:50.463286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:32:50.464397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:32:50.464397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:32:50.464397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:32:50.464397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:32:50.464397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:32:50.464397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:32:50.464397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:32:50.464397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:32:50.464397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:32:50.464397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:32:50.464397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:32:50.464397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:32:50.464397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:32:50.464397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:32:50.464397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:32:50.464397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:32:50.465409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:32:50.465409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:32:50.465409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:32:50.465409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:32:50.465409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:32:50.465409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:32:50.465409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:32:50.465409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:32:50.465409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:32:50.465409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:32:50.465409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:32:50.465409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:32:50.465409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 598/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:32:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:32:50.471011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2231f3-1144-40d5-adb5-427a440a607d None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:50.573531 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2231f3-1144-40d5-adb5-427a440a607d None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:50.608683 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6184f-6e3d-423c-977f-8dd972a90cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:50.609379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6184f-6e3d-423c-977f-8dd972a90cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:50.609656 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6184f-6e3d-423c-977f-8dd972a90cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:50.609926 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6184f-6e3d-423c-977f-8dd972a90cd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:50.610669 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6184f-6e3d-423c-977f-8dd972a90cd7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:50.610973 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6184f-6e3d-423c-977f-8dd972a90cd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:50.612333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc6184f-6e3d-423c-977f-8dd972a90cd7 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:50.617308 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:50.620580 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 591/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:32:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:50.627017 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a3179-e9b8-43ab-bf92-beab2a929120 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:50.650132 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:50.732262 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a3179-e9b8-43ab-bf92-beab2a929120 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:50.766890 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7384fc1-5e71-4d30-a009-0f91750aace0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:50.767756 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7384fc1-5e71-4d30-a009-0f91750aace0 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:50.768043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7384fc1-5e71-4d30-a009-0f91750aace0 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:50.768322 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7384fc1-5e71-4d30-a009-0f91750aace0 None admin] PATH_INFO: `/v3/users/14d65c39f9e344bea3244c453d38a8b9` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:50.769589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7384fc1-5e71-4d30-a009-0f91750aace0 None admin] RBAC: Authorizing `identity:get_user(user_id=14d65c39f9e344bea3244c453d38a8b9)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:50.771036 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7384fc1-5e71-4d30-a009-0f91750aace0 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:50.772662 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 599/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:32:50 2022] GET /identity/v3/users/14d65c39f9e344bea3244c453d38a8b9 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:50.778471 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6184f-6e3d-423c-977f-8dd972a90cd7 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:50.881458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc6184f-6e3d-423c-977f-8dd972a90cd7 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:50.916038 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8c8dc-0258-4794-8125-2b563d9af189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:50.916898 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8c8dc-0258-4794-8125-2b563d9af189 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:50.917214 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8c8dc-0258-4794-8125-2b563d9af189 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:50.917491 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8c8dc-0258-4794-8125-2b563d9af189 None admin] PATH_INFO: `/v3/projects/6f9e46a6cd6b4682b3e3c29c1ac06cd1` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:50.918681 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8c8dc-0258-4794-8125-2b563d9af189 None admin] RBAC: Authorizing `identity:get_project(project_id=6f9e46a6cd6b4682b3e3c29c1ac06cd1)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:50.920126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8c8dc-0258-4794-8125-2b563d9af189 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:50.922360 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 592/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:32:50 2022] GET /identity/v3/projects/6f9e46a6cd6b4682b3e3c29c1ac06cd1 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:50.928396 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7384fc1-5e71-4d30-a009-0f91750aace0 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:51.016602 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:51.028257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7384fc1-5e71-4d30-a009-0f91750aace0 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:51.061371 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dd4ba0-e821-47ef-9047-736297c5da3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:51.063032 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd4ba0-e821-47ef-9047-736297c5da3c None admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:51.063559 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd4ba0-e821-47ef-9047-736297c5da3c None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:51.063862 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dd4ba0-e821-47ef-9047-736297c5da3c None admin] PATH_INFO: `/v3/projects/6f9e46a6cd6b4682b3e3c29c1ac06cd1/users/14d65c39f9e344bea3244c453d38a8b9/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:51.066509 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd4ba0-e821-47ef-9047-736297c5da3c None admin] RBAC: Authorizing `identity:create_grant(project_id=6f9e46a6cd6b4682b3e3c29c1ac06cd1, user_id=14d65c39f9e344bea3244c453d38a8b9, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:51.067949 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dd4ba0-e821-47ef-9047-736297c5da3c None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:51.137089 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 600/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:32:50 2022] PUT /identity/v3/projects/6f9e46a6cd6b4682b3e3c29c1ac06cd1/users/14d65c39f9e344bea3244c453d38a8b9/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:32:53.341688 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8b43c-d7fc-477d-993b-aa8f3afa84ec None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:53.342287 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8b43c-d7fc-477d-993b-aa8f3afa84ec None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:53.342287 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8b43c-d7fc-477d-993b-aa8f3afa84ec None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:53.344184 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 593/1193] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:32:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:32:53.349748 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b277b-bac5-4538-8780-3bb9e902ad38 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:53.350208 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b277b-bac5-4538-8780-3bb9e902ad38 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:53.350317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855b277b-bac5-4538-8780-3bb9e902ad38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:53.368343 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-855b277b-bac5-4538-8780-3bb9e902ad38 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:32:53.369566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-855b277b-bac5-4538-8780-3bb9e902ad38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:32:53.437700 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 601/1194] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:32:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:32:53.462487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8b43c-d7fc-477d-993b-aa8f3afa84ec None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:53.531556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:53.568569 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8b43c-d7fc-477d-993b-aa8f3afa84ec None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:53.602478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed62eaa-3ca0-4b22-8536-a6d14ac6f2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:53.603188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed62eaa-3ca0-4b22-8536-a6d14ac6f2dc None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:53.603471 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed62eaa-3ca0-4b22-8536-a6d14ac6f2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:53.603752 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed62eaa-3ca0-4b22-8536-a6d14ac6f2dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:53.617093 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed62eaa-3ca0-4b22-8536-a6d14ac6f2dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:32:53.627229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-8ed62eaa-3ca0-4b22-8536-a6d14ac6f2dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:32:53.627229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:32:53.627229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:32:53.627229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:32:53.627229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:32:53.627229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:32:53.627229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:32:53.627229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:32:53.627229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:32:53.627229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:32:53.627229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:32:53.627229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:32:53.627229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:32:53.627229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:32:53.627229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:32:53.627229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:32:53.628339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:32:53.628339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:32:53.628339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:32:53.628339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:32:53.628339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:32:53.628339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:32:53.628339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:32:53.628339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:32:53.628339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:32:53.628339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:32:53.628339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:32:53.628339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:32:53.628339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:32:53.628339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:32:53.628339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:32:53.628339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:32:53.629442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:32:53.629442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:32:53.629442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:32:53.629442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:32:53.629442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:32:53.629442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:32:53.629442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:32:53.629442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:32:53.629442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:32:53.629442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:32:53.629442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:32:53.629442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:32:53.629442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 594/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:32:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:32:53.636895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b277b-bac5-4538-8780-3bb9e902ad38 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:53.717181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:53.739140 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855b277b-bac5-4538-8780-3bb9e902ad38 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:53.774077 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd880f59-c904-4d78-b58f-24eb70d71acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:53.774773 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd880f59-c904-4d78-b58f-24eb70d71acf None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:53.775057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd880f59-c904-4d78-b58f-24eb70d71acf None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:53.775331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd880f59-c904-4d78-b58f-24eb70d71acf None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:53.776176 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd880f59-c904-4d78-b58f-24eb70d71acf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:53.776481 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd880f59-c904-4d78-b58f-24eb70d71acf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:53.777923 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd880f59-c904-4d78-b58f-24eb70d71acf None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:53.786087 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 602/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:32:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:53.792277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed62eaa-3ca0-4b22-8536-a6d14ac6f2dc None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:53.899620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed62eaa-3ca0-4b22-8536-a6d14ac6f2dc None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:53.926653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:53.934412 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80bb46e-867d-4c07-945f-748f3249d2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:53.935281 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bb46e-867d-4c07-945f-748f3249d2ab None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:53.935567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bb46e-867d-4c07-945f-748f3249d2ab None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:53.935847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bb46e-867d-4c07-945f-748f3249d2ab None admin] PATH_INFO: `/v3/users/14d65c39f9e344bea3244c453d38a8b9` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:53.938083 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80bb46e-867d-4c07-945f-748f3249d2ab None admin] RBAC: Authorizing `identity:get_user(user_id=14d65c39f9e344bea3244c453d38a8b9)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:53.939530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80bb46e-867d-4c07-945f-748f3249d2ab None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:53.945381 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 595/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Fri May 27 19:32:53 2022] GET /identity/v3/users/14d65c39f9e344bea3244c453d38a8b9 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:53.948572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd880f59-c904-4d78-b58f-24eb70d71acf None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:54.052272 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd880f59-c904-4d78-b58f-24eb70d71acf None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:54.086632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bcbe1-1c96-451c-bb96-d375bc25ef67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:54.087309 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bcbe1-1c96-451c-bb96-d375bc25ef67 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:54.087595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bcbe1-1c96-451c-bb96-d375bc25ef67 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:54.087872 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bcbe1-1c96-451c-bb96-d375bc25ef67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:54.099013 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:54.155797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186bcbe1-1c96-451c-bb96-d375bc25ef67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:54.157199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186bcbe1-1c96-451c-bb96-d375bc25ef67 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:54.251492 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 603/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:32:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 19:32:54.257414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80bb46e-867d-4c07-945f-748f3249d2ab None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:54.357396 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80bb46e-867d-4c07-945f-748f3249d2ab None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:54.395586 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03d6615-9f95-4061-a29f-393ba9878124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:54.396484 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d6615-9f95-4061-a29f-393ba9878124 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:54.396761 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d6615-9f95-4061-a29f-393ba9878124 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:54.397055 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03d6615-9f95-4061-a29f-393ba9878124 None admin] PATH_INFO: `/v3/projects/6f9e46a6cd6b4682b3e3c29c1ac06cd1` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:54.398226 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d6615-9f95-4061-a29f-393ba9878124 None admin] RBAC: Authorizing `identity:get_project(project_id=6f9e46a6cd6b4682b3e3c29c1ac06cd1)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:54.399698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03d6615-9f95-4061-a29f-393ba9878124 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:54.401412 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 596/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Fri May 27 19:32:54 2022] GET /identity/v3/projects/6f9e46a6cd6b4682b3e3c29c1ac06cd1 => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:32:54.408503 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bcbe1-1c96-451c-bb96-d375bc25ef67 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:32:54.449660 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:32:54.508756 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bcbe1-1c96-451c-bb96-d375bc25ef67 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:32:54.544088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c134d-ba80-494b-9297-61b765754cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:32:54.544745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c134d-ba80-494b-9297-61b765754cdc None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:32:54.545061 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c134d-ba80-494b-9297-61b765754cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:32:54.545981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c134d-ba80-494b-9297-61b765754cdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:32:54.546858 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c134d-ba80-494b-9297-61b765754cdc None admin] RBAC: Adding query filter params (role.id=6505bbb129404436966ee59aa7a01d8e, scope.project.id=6f9e46a6cd6b4682b3e3c29c1ac06cd1, user.id=14d65c39f9e344bea3244c453d38a8b9) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:32:54.547179 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c134d-ba80-494b-9297-61b765754cdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:32:54.548607 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c134d-ba80-494b-9297-61b765754cdc None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:32:54.554566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 604/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:32:54 2022] GET /identity/v3/role_assignments?user.id=14d65c39f9e344bea3244c453d38a8b9&scope.project.id=6f9e46a6cd6b4682b3e3c29c1ac06cd1&role.id=6505bbb129404436966ee59aa7a01d8e => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:33:04.391298 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=68 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:33:04.551766 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=48 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:15.176353 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28cb0c-e735-42b3-8b0a-d0d0c2a22aa5 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:15.177279 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28cb0c-e735-42b3-8b0a-d0d0c2a22aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:15.177279 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff28cb0c-e735-42b3-8b0a-d0d0c2a22aa5 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:15.178913 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 597/1201] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:34:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:34:15.185726 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf9895-b1c7-4a4b-a9f7-00de78a95fb1 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:15.196927 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf9895-b1c7-4a4b-a9f7-00de78a95fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:15.196927 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf9895-b1c7-4a4b-a9f7-00de78a95fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:15.203717 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-0bdf9895-b1c7-4a4b-a9f7-00de78a95fb1 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:34:15.204925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-0bdf9895-b1c7-4a4b-a9f7-00de78a95fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:34:15.278117 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 605/1202] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:34:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 19:34:15.303724 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28cb0c-e735-42b3-8b0a-d0d0c2a22aa5 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:15.412204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff28cb0c-e735-42b3-8b0a-d0d0c2a22aa5 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:15.448510 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249df5c6-4021-40c9-9f9c-704b0bcbd2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:15.449409 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249df5c6-4021-40c9-9f9c-704b0bcbd2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:15.449676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249df5c6-4021-40c9-9f9c-704b0bcbd2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:15.449952 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249df5c6-4021-40c9-9f9c-704b0bcbd2a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:15.460607 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249df5c6-4021-40c9-9f9c-704b0bcbd2a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:15.462350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249df5c6-4021-40c9-9f9c-704b0bcbd2a4 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:15.480235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-249df5c6-4021-40c9-9f9c-704b0bcbd2a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:15.480235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:15.480235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:15.480235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:15.480235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:15.480235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:15.480235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:15.480235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:15.480235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:15.480235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:15.480235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:15.480235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:15.480235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:34:15.480235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:34:15.480235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:34:15.484087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:34:15.484087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:15.484087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:15.484087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:15.484087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:15.484087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:15.484087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:15.484087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:15.484087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:34:15.484087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:15.484087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:15.484087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:15.484087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:15.484087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:15.484087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:15.484087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:15.484087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:15.485715 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:34:15.485715 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:34:15.485715 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:15.485715 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:34:15.485715 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 598/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:34:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:15.487724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf9895-b1c7-4a4b-a9f7-00de78a95fb1 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:15.591999 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf9895-b1c7-4a4b-a9f7-00de78a95fb1 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:15.621064 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:15.626902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfebb873-aa81-4e75-ad81-4bd5c54b36ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:15.628661 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebb873-aa81-4e75-ad81-4bd5c54b36ac None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:15.628661 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebb873-aa81-4e75-ad81-4bd5c54b36ac None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:15.628661 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfebb873-aa81-4e75-ad81-4bd5c54b36ac None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:15.628939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfebb873-aa81-4e75-ad81-4bd5c54b36ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:15.629227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfebb873-aa81-4e75-ad81-4bd5c54b36ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:15.630621 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfebb873-aa81-4e75-ad81-4bd5c54b36ac None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:15.642167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 606/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:15.647915 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249df5c6-4021-40c9-9f9c-704b0bcbd2a4 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:15.700697 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:15.750994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249df5c6-4021-40c9-9f9c-704b0bcbd2a4 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:15.783765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc05fac-92c8-4150-9e66-dd817e4a2eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:15.784714 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc05fac-92c8-4150-9e66-dd817e4a2eb7 None admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:15.785214 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc05fac-92c8-4150-9e66-dd817e4a2eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:15.786130 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc05fac-92c8-4150-9e66-dd817e4a2eb7 None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:15.786937 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc05fac-92c8-4150-9e66-dd817e4a2eb7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:15.788424 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc05fac-92c8-4150-9e66-dd817e4a2eb7 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:15.894514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 599/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Fri May 27 19:34:15 2022] POST /identity/v3/users => generated 251 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:34:18.158650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147dd431-c947-4af3-894c-2479e6570b1f None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:18.159312 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147dd431-c947-4af3-894c-2479e6570b1f None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:18.159312 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147dd431-c947-4af3-894c-2479e6570b1f None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:18.160580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 607/1206] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:34:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:34:18.165769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f35af-7423-45dc-a052-abb604cef619 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:18.166218 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f35af-7423-45dc-a052-abb604cef619 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:18.166317 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f35af-7423-45dc-a052-abb604cef619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:18.184086 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-e84f35af-7423-45dc-a052-abb604cef619 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:34:18.185327 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-e84f35af-7423-45dc-a052-abb604cef619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:34:18.255194 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 600/1207] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:34:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:34:18.282651 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147dd431-c947-4af3-894c-2479e6570b1f None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:18.385641 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147dd431-c947-4af3-894c-2479e6570b1f None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:18.427476 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58641564-8851-4892-9512-2b2c78ee9e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:18.427476 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58641564-8851-4892-9512-2b2c78ee9e35 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:18.427476 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58641564-8851-4892-9512-2b2c78ee9e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:18.427476 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58641564-8851-4892-9512-2b2c78ee9e35 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:18.431685 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58641564-8851-4892-9512-2b2c78ee9e35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:34:18.439388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-58641564-8851-4892-9512-2b2c78ee9e35 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 19:34:18.439388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:18.439388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:18.439388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:18.439388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:18.439388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:18.439388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:18.439388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:18.439388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:18.439388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:18.439388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:18.439388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:18.439388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:34:18.439388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:34:18.439388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:34:18.439388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:34:18.440566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:34:18.440566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:34:18.440566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:34:18.440566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:34:18.440566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:18.440566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:18.440566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:18.440566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:18.440566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:18.440566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:18.440566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:18.440566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:34:18.440566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:18.440566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:18.440566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:18.440566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:18.441663 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:18.441663 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:18.441663 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:18.441663 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:18.441663 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:34:18.441663 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:34:18.441663 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:34:18.441663 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:34:18.441663 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:34:18.441663 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:34:18.441663 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 19:34:18.441663 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:18.441663 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 608/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:34:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:18.447726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f35af-7423-45dc-a052-abb604cef619 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:18.490240 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:18.551003 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f35af-7423-45dc-a052-abb604cef619 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:18.589526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead4876-9896-42c3-b472-e72fba8569d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:18.593860 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead4876-9896-42c3-b472-e72fba8569d3 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:18.593860 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead4876-9896-42c3-b472-e72fba8569d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:18.593860 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ead4876-9896-42c3-b472-e72fba8569d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:18.593860 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ead4876-9896-42c3-b472-e72fba8569d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:18.593860 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ead4876-9896-42c3-b472-e72fba8569d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:18.593860 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ead4876-9896-42c3-b472-e72fba8569d3 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:18.601272 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 601/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Fri May 27 19:34:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:18.607221 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58641564-8851-4892-9512-2b2c78ee9e35 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:18.658907 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:18.710311 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58641564-8851-4892-9512-2b2c78ee9e35 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:18.745829 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e943d63-a4ff-4795-9f35-46c5e060777b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:18.746625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e943d63-a4ff-4795-9f35-46c5e060777b None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:18.746913 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e943d63-a4ff-4795-9f35-46c5e060777b None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:18.747194 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e943d63-a4ff-4795-9f35-46c5e060777b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:18.757743 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e943d63-a4ff-4795-9f35-46c5e060777b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:18.759952 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e943d63-a4ff-4795-9f35-46c5e060777b None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:18.778053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-7e943d63-a4ff-4795-9f35-46c5e060777b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:18.778053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:18.778053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:18.778053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:18.778053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:18.778053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:18.778053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:18.778053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:18.778053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:18.778053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:18.778053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:18.778053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:18.778053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:34:18.778053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:34:18.778053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:34:18.779181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:34:18.779181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:18.779181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:18.779181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:18.779181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:18.779181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:18.779181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:18.779181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:18.779181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:34:18.779181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:18.779181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:18.779181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:18.779181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:18.779181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:18.779181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:18.779181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:18.779181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:18.780344 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:34:18.780344 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:34:18.780344 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:18.780344 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:18.780344 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 609/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:34:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:18.785205 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead4876-9896-42c3-b472-e72fba8569d3 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:18.883498 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:18.887040 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ead4876-9896-42c3-b472-e72fba8569d3 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:18.923909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d32ece-4125-47da-9403-459a5d1c1f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:18.924597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d32ece-4125-47da-9403-459a5d1c1f50 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:18.924878 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d32ece-4125-47da-9403-459a5d1c1f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:18.925192 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d32ece-4125-47da-9403-459a5d1c1f50 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:18.925928 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d32ece-4125-47da-9403-459a5d1c1f50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:18.926295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d32ece-4125-47da-9403-459a5d1c1f50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:18.927728 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d32ece-4125-47da-9403-459a5d1c1f50 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:18.940835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 602/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:34:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:18.947266 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e943d63-a4ff-4795-9f35-46c5e060777b None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:19.055313 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e943d63-a4ff-4795-9f35-46c5e060777b None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:19.057487 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:19.091629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24faad5f-01f1-40c4-9893-cf6b07a83e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:19.092434 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24faad5f-01f1-40c4-9893-cf6b07a83e35 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:19.092717 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24faad5f-01f1-40c4-9893-cf6b07a83e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:19.093224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24faad5f-01f1-40c4-9893-cf6b07a83e35 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:19.100451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24faad5f-01f1-40c4-9893-cf6b07a83e35 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:19.101944 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24faad5f-01f1-40c4-9893-cf6b07a83e35 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:19.118458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-24faad5f-01f1-40c4-9893-cf6b07a83e35 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 19:34:19.118458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:19.118458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:19.118458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:19.118458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:19.118458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:19.118458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:19.118458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:19.118458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:19.118458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:19.118458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:19.118458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:19.118458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:34:19.118458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:34:19.118458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:34:19.118458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:34:19.120333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:19.120333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:19.120333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:34:19.120333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:34:19.120333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:34:19.120333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:34:19.120333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:19.120333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:19.120333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:19.120333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:19.120333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:19.120333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:34:19.120333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:19.120333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:19.120333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:19.120333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:19.120333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:19.120333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:19.121613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:19.121613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:19.121613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:34:19.121613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:34:19.121613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:34:19.121613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:34:19.121613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:34:19.121613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:34:19.121613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 19:34:19.121613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:19.121613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 610/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:34:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:19.126289 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d32ece-4125-47da-9403-459a5d1c1f50 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:19.232145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d32ece-4125-47da-9403-459a5d1c1f50 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:19.269250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4c400-9c5b-4c09-b3c7-d180bdccf9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:19.270016 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4c400-9c5b-4c09-b3c7-d180bdccf9ac None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:19.270445 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4c400-9c5b-4c09-b3c7-d180bdccf9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:19.270723 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4c400-9c5b-4c09-b3c7-d180bdccf9ac None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:19.271561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4c400-9c5b-4c09-b3c7-d180bdccf9ac None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:19.271870 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4c400-9c5b-4c09-b3c7-d180bdccf9ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:19.273321 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4c400-9c5b-4c09-b3c7-d180bdccf9ac None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:19.294179 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 603/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Fri May 27 19:34:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:19.300002 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24faad5f-01f1-40c4-9893-cf6b07a83e35 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:19.414355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24faad5f-01f1-40c4-9893-cf6b07a83e35 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:19.453802 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72ff5c-71f2-442b-9e43-2c6a7c19c43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:19.454456 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72ff5c-71f2-442b-9e43-2c6a7c19c43c None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:19.454746 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72ff5c-71f2-442b-9e43-2c6a7c19c43c None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:19.455027 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c72ff5c-71f2-442b-9e43-2c6a7c19c43c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:19.493096 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:19.540078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72ff5c-71f2-442b-9e43-2c6a7c19c43c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:19.547851 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c72ff5c-71f2-442b-9e43-2c6a7c19c43c None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:19.655503 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 611/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:34:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 19:34:19.661445 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4c400-9c5b-4c09-b3c7-d180bdccf9ac None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:19.680716 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:19.776514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4c400-9c5b-4c09-b3c7-d180bdccf9ac None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:19.810829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bdb1c6-c5f3-4d75-8282-2ce8fec3dda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:19.811627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdb1c6-c5f3-4d75-8282-2ce8fec3dda5 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:19.811913 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdb1c6-c5f3-4d75-8282-2ce8fec3dda5 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:19.812191 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdb1c6-c5f3-4d75-8282-2ce8fec3dda5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:19.823488 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bdb1c6-c5f3-4d75-8282-2ce8fec3dda5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:19.829411 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bdb1c6-c5f3-4d75-8282-2ce8fec3dda5 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:19.843942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-94bdb1c6-c5f3-4d75-8282-2ce8fec3dda5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:19.843942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:19.843942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:19.843942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:19.843942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:19.843942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:19.843942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:19.843942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:19.843942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:19.843942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:19.843942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:19.843942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:19.843942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:34:19.843942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:34:19.843942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:34:19.845081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:34:19.845081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:19.845081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:19.845081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:19.845081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:19.845081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:19.845081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:19.845081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:19.845081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:34:19.845081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:19.845081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:19.845081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:19.845081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:19.845081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:19.845081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:19.845081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:19.845081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:19.846086 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:34:19.846086 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:34:19.846086 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:19.846086 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:34:19.846086 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 604/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:34:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:19.851027 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72ff5c-71f2-442b-9e43-2c6a7c19c43c None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:19.916450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:19.963440 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c72ff5c-71f2-442b-9e43-2c6a7c19c43c None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:20.005016 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc35cde-526d-4544-914f-ce0f8732112d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:20.005687 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc35cde-526d-4544-914f-ce0f8732112d None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:20.005972 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc35cde-526d-4544-914f-ce0f8732112d None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:20.006251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc35cde-526d-4544-914f-ce0f8732112d None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:20.006978 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc35cde-526d-4544-914f-ce0f8732112d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:20.007288 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc35cde-526d-4544-914f-ce0f8732112d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:20.008670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc35cde-526d-4544-914f-ce0f8732112d None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:20.021403 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 612/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:34:20.027008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bdb1c6-c5f3-4d75-8282-2ce8fec3dda5 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:20.106424 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:20.139051 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bdb1c6-c5f3-4d75-8282-2ce8fec3dda5 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:20.177419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc3cd5-e569-4d05-859d-ad28f5ca6280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:20.178220 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc3cd5-e569-4d05-859d-ad28f5ca6280 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:20.178497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc3cd5-e569-4d05-859d-ad28f5ca6280 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:20.178799 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc3cd5-e569-4d05-859d-ad28f5ca6280 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:20.186796 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc3cd5-e569-4d05-859d-ad28f5ca6280 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:20.188719 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc3cd5-e569-4d05-859d-ad28f5ca6280 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:20.203236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-56fc3cd5-e569-4d05-859d-ad28f5ca6280 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:34:20.203236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:20.203236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:20.203236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:20.203236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:20.203236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:20.203236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:20.203236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:20.203236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:20.203236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:20.203236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:20.203236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:20.203236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:34:20.203236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:34:20.203236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:34:20.204295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:34:20.204295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:20.204295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:20.204295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:20.204295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:20.204295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:20.204295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:20.204295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:20.204295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:34:20.204295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:20.204295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:20.204295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:20.204295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:20.204295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:20.204295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:20.204295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:20.204295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:20.205385 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:34:20.205385 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:34:20.205385 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:34:20.205385 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:34:20.205385 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:34:20.205385 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:34:20.205385 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:34:20.205385 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:34:20.205385 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 605/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:34:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:34:20.210455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc35cde-526d-4544-914f-ce0f8732112d None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:20.312435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc35cde-526d-4544-914f-ce0f8732112d None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:20.336606 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:20.349326 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410e7c5-d28a-47b4-9edc-4e31fbeab66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:20.350021 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410e7c5-d28a-47b4-9edc-4e31fbeab66e None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:20.350309 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410e7c5-d28a-47b4-9edc-4e31fbeab66e None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:20.350589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410e7c5-d28a-47b4-9edc-4e31fbeab66e None admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:20.351382 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410e7c5-d28a-47b4-9edc-4e31fbeab66e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:20.351696 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410e7c5-d28a-47b4-9edc-4e31fbeab66e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:20.353360 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a410e7c5-d28a-47b4-9edc-4e31fbeab66e None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:20.365622 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 613/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:34:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:20.373184 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc3cd5-e569-4d05-859d-ad28f5ca6280 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:20.476399 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc3cd5-e569-4d05-859d-ad28f5ca6280 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:20.507850 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:20.513932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0294a156-09d9-4155-a273-a60307abb480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:20.516309 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294a156-09d9-4155-a273-a60307abb480 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:20.516309 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294a156-09d9-4155-a273-a60307abb480 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:20.516309 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0294a156-09d9-4155-a273-a60307abb480 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:20.516309 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0294a156-09d9-4155-a273-a60307abb480 None admin] RBAC: Adding query filter params (role.id=11e7404599ae49afb5edd22dabe1a5b2, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=ffc2ed13a0c84f128b795002892efc2f) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:20.516309 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0294a156-09d9-4155-a273-a60307abb480 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:20.517811 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0294a156-09d9-4155-a273-a60307abb480 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:20.526811 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 606/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:34:20 2022] GET /identity/v3/role_assignments?user.id=ffc2ed13a0c84f128b795002892efc2f&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=11e7404599ae49afb5edd22dabe1a5b2 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:34:22.744607 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684b8ff-8fc5-4b6f-aab5-b7967ead3f0b None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:22.745203 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684b8ff-8fc5-4b6f-aab5-b7967ead3f0b None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:22.745203 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684b8ff-8fc5-4b6f-aab5-b7967ead3f0b None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:22.746720 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 614/1220] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:34:22.752249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857d756-b43a-455e-9c68-243762f02e55 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:22.752756 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857d756-b43a-455e-9c68-243762f02e55 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:22.752874 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1857d756-b43a-455e-9c68-243762f02e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:22.770354 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-1857d756-b43a-455e-9c68-243762f02e55 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:34:22.771465 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-1857d756-b43a-455e-9c68-243762f02e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:34:22.848603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 607/1221] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:34:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 19:34:22.871225 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684b8ff-8fc5-4b6f-aab5-b7967ead3f0b None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:22.977367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684b8ff-8fc5-4b6f-aab5-b7967ead3f0b None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:23.012624 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62673e6b-02fb-4d8a-bb72-92677a73a9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:23.013458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62673e6b-02fb-4d8a-bb72-92677a73a9ba None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:23.013747 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62673e6b-02fb-4d8a-bb72-92677a73a9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:23.014020 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62673e6b-02fb-4d8a-bb72-92677a73a9ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:23.025906 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62673e6b-02fb-4d8a-bb72-92677a73a9ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:34:23.032657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-62673e6b-02fb-4d8a-bb72-92677a73a9ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 19:34:23.032657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:23.032657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:23.032657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:23.032657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:23.032657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:23.032657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:23.032657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:23.032657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:23.032657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:23.032657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:23.032657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:23.032657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:34:23.032657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:34:23.032657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:34:23.032657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:34:23.033860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:34:23.033860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:34:23.033860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:34:23.033860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:34:23.033860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:23.033860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:23.033860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:23.033860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:23.033860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:23.033860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:23.033860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:23.033860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:34:23.033860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:23.033860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:23.033860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:23.033860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:23.034926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:23.034926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:23.034926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:23.034926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:23.034926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:34:23.034926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:34:23.034926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:34:23.034926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:34:23.034926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:34:23.034926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:34:23.034926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 19:34:23.034926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:23.034926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 615/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:34:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:23.040618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857d756-b43a-455e-9c68-243762f02e55 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:23.143367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1857d756-b43a-455e-9c68-243762f02e55 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:23.167830 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:23.178271 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b463a8-b648-4ebb-b024-e1e7977d151c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:23.178950 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b463a8-b648-4ebb-b024-e1e7977d151c None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:23.179230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b463a8-b648-4ebb-b024-e1e7977d151c None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:23.179505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b463a8-b648-4ebb-b024-e1e7977d151c None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:23.180255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b463a8-b648-4ebb-b024-e1e7977d151c None admin] RBAC: Adding query filter params (name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:23.180562 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b463a8-b648-4ebb-b024-e1e7977d151c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:23.182813 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b463a8-b648-4ebb-b024-e1e7977d151c None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:23.190746 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 608/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Fri May 27 19:34:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:23.196875 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62673e6b-02fb-4d8a-bb72-92677a73a9ba None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:23.241370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:23.298315 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62673e6b-02fb-4d8a-bb72-92677a73a9ba None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:23.332943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd9b3a-300c-45e4-ae0e-c1ec4abcbce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:23.333772 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd9b3a-300c-45e4-ae0e-c1ec4abcbce1 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:23.334064 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd9b3a-300c-45e4-ae0e-c1ec4abcbce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:23.334341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd9b3a-300c-45e4-ae0e-c1ec4abcbce1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:23.346040 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd9b3a-300c-45e4-ae0e-c1ec4abcbce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:23.347521 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd9b3a-300c-45e4-ae0e-c1ec4abcbce1 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:23.365573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-4fdd9b3a-300c-45e4-ae0e-c1ec4abcbce1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:23.365573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:23.365573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:23.365573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:23.365573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:23.365573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:23.365573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:23.365573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:23.365573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:23.365573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:23.365573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:23.365573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:23.365573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:34:23.365573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:34:23.365573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:34:23.366719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:34:23.366719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:23.366719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:23.366719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:23.366719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:23.366719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:23.366719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:23.366719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:23.366719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:34:23.366719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:23.366719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:23.366719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:23.366719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:23.366719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:23.366719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:23.366719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:23.366719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:23.367990 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:34:23.367990 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:34:23.367990 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:23.367990 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:23.367990 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 616/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:34:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:23.372923 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b463a8-b648-4ebb-b024-e1e7977d151c None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:23.473799 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b463a8-b648-4ebb-b024-e1e7977d151c None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:23.506909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c16f45-11a7-468f-b17e-89c075935d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:23.507578 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c16f45-11a7-468f-b17e-89c075935d59 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:23.507859 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c16f45-11a7-468f-b17e-89c075935d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:23.508133 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c16f45-11a7-468f-b17e-89c075935d59 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:23.508881 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c16f45-11a7-468f-b17e-89c075935d59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:23.509252 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c16f45-11a7-468f-b17e-89c075935d59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:23.511259 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c16f45-11a7-468f-b17e-89c075935d59 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:23.523603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 609/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:23.530731 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd9b3a-300c-45e4-ae0e-c1ec4abcbce1 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:23.624559 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:23.631191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd9b3a-300c-45e4-ae0e-c1ec4abcbce1 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:23.664809 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0ec2e-7efd-4bd0-8c63-43038b87a8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:23.665632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0ec2e-7efd-4bd0-8c63-43038b87a8ed None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:23.665937 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0ec2e-7efd-4bd0-8c63-43038b87a8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:23.666214 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0ec2e-7efd-4bd0-8c63-43038b87a8ed None admin] PATH_INFO: `/v3/users/placement` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:23.673023 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0ec2e-7efd-4bd0-8c63-43038b87a8ed None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:23.674468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0ec2e-7efd-4bd0-8c63-43038b87a8ed None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:23.688943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-03f0ec2e-7efd-4bd0-8c63-43038b87a8ed None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 19:34:23.688943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:23.688943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:23.688943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:23.688943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:23.688943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:23.688943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:23.688943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:23.688943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:23.688943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:23.688943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:23.688943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:23.688943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:34:23.688943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:34:23.688943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:34:23.688943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:34:23.690120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:23.690120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:23.690120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:34:23.690120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:34:23.690120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:34:23.690120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:34:23.690120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:23.690120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:23.690120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:23.690120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:23.690120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:23.690120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:34:23.690120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:23.690120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:23.690120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:23.690120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:23.690120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:23.690120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:23.691105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:23.691105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:23.691105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:34:23.691105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:34:23.691105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:34:23.691105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:34:23.691105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:34:23.691105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:34:23.691105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 19:34:23.691105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:23.691105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 617/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:34:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:23.696913 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c16f45-11a7-468f-b17e-89c075935d59 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:23.746301 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:23.800011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c16f45-11a7-468f-b17e-89c075935d59 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:23.834644 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34780061-926f-461f-b5b8-4609c3417fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:23.835317 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34780061-926f-461f-b5b8-4609c3417fae None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:23.835602 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34780061-926f-461f-b5b8-4609c3417fae None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:23.835882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34780061-926f-461f-b5b8-4609c3417fae None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:23.836923 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34780061-926f-461f-b5b8-4609c3417fae None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:23.837272 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34780061-926f-461f-b5b8-4609c3417fae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:23.839050 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34780061-926f-461f-b5b8-4609c3417fae None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:23.858314 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 610/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Fri May 27 19:34:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:23.864641 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0ec2e-7efd-4bd0-8c63-43038b87a8ed None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:23.980690 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0ec2e-7efd-4bd0-8c63-43038b87a8ed None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:24.021004 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d24b6-5d5b-4342-8dcf-d42eec5076d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:24.021816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d24b6-5d5b-4342-8dcf-d42eec5076d4 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:24.022095 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d24b6-5d5b-4342-8dcf-d42eec5076d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:24.022390 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d24b6-5d5b-4342-8dcf-d42eec5076d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:24.033732 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d24b6-5d5b-4342-8dcf-d42eec5076d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:24.035199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d24b6-5d5b-4342-8dcf-d42eec5076d4 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:24.053227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-684d24b6-5d5b-4342-8dcf-d42eec5076d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:24.053227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:24.053227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:24.053227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:24.053227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:24.053227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:24.053227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:24.053227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:24.053227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:24.053227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:24.053227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:24.053227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:24.053227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:34:24.053227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:34:24.053227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:34:24.054363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:34:24.054363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:24.054363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:24.054363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:24.054363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:24.054363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:24.054363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:24.054363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:24.054363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:34:24.054363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:24.054363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:24.054363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:24.054363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:24.054363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:24.054363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:24.054363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:24.054363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:24.055554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:34:24.055554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:34:24.055554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:24.055554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:24.055554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 618/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:34:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:24.061402 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34780061-926f-461f-b5b8-4609c3417fae None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:24.162395 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34780061-926f-461f-b5b8-4609c3417fae None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:24.164662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:24.196976 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f25a3-864f-4afb-9328-e50c0be15828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:24.197951 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f25a3-864f-4afb-9328-e50c0be15828 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:24.198217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f25a3-864f-4afb-9328-e50c0be15828 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:24.198495 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f25a3-864f-4afb-9328-e50c0be15828 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:24.199220 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f25a3-864f-4afb-9328-e50c0be15828 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:24.199522 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f25a3-864f-4afb-9328-e50c0be15828 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:24.201013 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f25a3-864f-4afb-9328-e50c0be15828 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:24.212230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 611/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:34:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:24.218400 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d24b6-5d5b-4342-8dcf-d42eec5076d4 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:24.227827 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:24.317707 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d24b6-5d5b-4342-8dcf-d42eec5076d4 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:24.351287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540f8c7-676e-4375-9875-1690352190d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:24.352076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540f8c7-676e-4375-9875-1690352190d6 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:24.352359 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540f8c7-676e-4375-9875-1690352190d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:24.352634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9540f8c7-676e-4375-9875-1690352190d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:24.359954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540f8c7-676e-4375-9875-1690352190d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:24.361431 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9540f8c7-676e-4375-9875-1690352190d6 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:24.376446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-9540f8c7-676e-4375-9875-1690352190d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:34:24.376446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:24.376446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:24.376446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:24.376446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:24.376446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:24.376446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:24.376446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:24.376446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:24.376446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:24.376446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:24.376446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:24.376446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:34:24.376446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:34:24.376446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:34:24.380141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:34:24.380141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:24.380141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:24.380141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:24.380141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:24.380141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:24.380141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:24.380141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:24.380141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:34:24.380141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:24.380141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:24.380141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:24.380141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:24.380141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:24.380141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:24.380141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:24.380141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:24.381303 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:34:24.381303 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:34:24.381303 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:34:24.381303 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:34:24.381303 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:34:24.381303 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:34:24.381303 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:34:24.381303 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:24.381303 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 619/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:34:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:24.383534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f25a3-864f-4afb-9328-e50c0be15828 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:24.485306 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f25a3-864f-4afb-9328-e50c0be15828 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:24.520231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee823d1f-61b2-4a0e-a447-1484150f04c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:24.520904 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee823d1f-61b2-4a0e-a447-1484150f04c1 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:24.521342 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee823d1f-61b2-4a0e-a447-1484150f04c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:24.521623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee823d1f-61b2-4a0e-a447-1484150f04c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:24.522413 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee823d1f-61b2-4a0e-a447-1484150f04c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:24.522723 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee823d1f-61b2-4a0e-a447-1484150f04c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:24.524171 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee823d1f-61b2-4a0e-a447-1484150f04c1 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:24.537170 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 612/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:34:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:24.543236 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540f8c7-676e-4375-9875-1690352190d6 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:24.610829 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:24.645432 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9540f8c7-676e-4375-9875-1690352190d6 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:24.677250 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a5b1c-cf94-45f8-bc7d-6f76c9d8b4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:24.678134 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a5b1c-cf94-45f8-bc7d-6f76c9d8b4fb None admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:24.678415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a5b1c-cf94-45f8-bc7d-6f76c9d8b4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:24.678692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a5b1c-cf94-45f8-bc7d-6f76c9d8b4fb None admin] PATH_INFO: `/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/ffc2ed13a0c84f128b795002892efc2f/roles/11e7404599ae49afb5edd22dabe1a5b2` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:24.692799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a5b1c-cf94-45f8-bc7d-6f76c9d8b4fb None admin] RBAC: Authorizing `identity:create_grant(project_id=a47d6b6971c3452e9799e8146856bdc5, user_id=ffc2ed13a0c84f128b795002892efc2f, role_id=11e7404599ae49afb5edd22dabe1a5b2)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:24.694276 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a5b1c-cf94-45f8-bc7d-6f76c9d8b4fb None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:24.815735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 620/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:34:24 2022] PUT /identity/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/ffc2ed13a0c84f128b795002892efc2f/roles/11e7404599ae49afb5edd22dabe1a5b2 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:34:27.132898 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988faab-ed14-441e-8c18-aca1af547918 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:27.133465 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988faab-ed14-441e-8c18-aca1af547918 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:27.133465 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e988faab-ed14-441e-8c18-aca1af547918 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:27.134885 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 613/1233] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:34:27.140979 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640fa14-72a7-411d-9f33-ff7b2bb8ddd6 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:27.141437 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640fa14-72a7-411d-9f33-ff7b2bb8ddd6 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:27.141673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640fa14-72a7-411d-9f33-ff7b2bb8ddd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:27.158531 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-4640fa14-72a7-411d-9f33-ff7b2bb8ddd6 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:34:27.159729 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-4640fa14-72a7-411d-9f33-ff7b2bb8ddd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:34:27.229613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 621/1234] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:34:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:34:27.253643 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e988faab-ed14-441e-8c18-aca1af547918 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:27.282541 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:27.359495 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e988faab-ed14-441e-8c18-aca1af547918 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:27.395058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68ac45-dfe9-4fef-aba6-5fe1dd24ce54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:27.395776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68ac45-dfe9-4fef-aba6-5fe1dd24ce54 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:27.396060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68ac45-dfe9-4fef-aba6-5fe1dd24ce54 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:27.396392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68ac45-dfe9-4fef-aba6-5fe1dd24ce54 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:27.407346 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68ac45-dfe9-4fef-aba6-5fe1dd24ce54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:34:27.415028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-3e68ac45-dfe9-4fef-aba6-5fe1dd24ce54 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 19:34:27.415028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:27.415028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:27.415028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:27.415028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:27.415028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:27.415028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:27.415028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:27.415028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:27.415028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:27.415028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:27.415028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:27.415028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:34:27.415028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:34:27.415028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:34:27.415028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:34:27.423305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:34:27.423305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:34:27.423305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:34:27.423305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:34:27.423305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:27.423305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:27.423305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:27.423305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:27.423305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:27.423305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:27.423305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:27.423305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:34:27.423305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:27.423305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:27.423305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:27.423305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:27.424350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:27.424350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:27.424350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:27.424350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:27.424350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:34:27.424350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:34:27.424350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:34:27.424350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:34:27.424350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:34:27.424350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:34:27.424350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 19:34:27.424350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:34:27.424350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 614/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Fri May 27 19:34:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:27.425262 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640fa14-72a7-411d-9f33-ff7b2bb8ddd6 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:27.470482 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:27.528971 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640fa14-72a7-411d-9f33-ff7b2bb8ddd6 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:27.564060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ded52a-f57a-4d6a-bfd5-55b6d0e221e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:27.564726 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ded52a-f57a-4d6a-bfd5-55b6d0e221e8 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:27.565047 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ded52a-f57a-4d6a-bfd5-55b6d0e221e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:27.565329 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ded52a-f57a-4d6a-bfd5-55b6d0e221e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:27.566081 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ded52a-f57a-4d6a-bfd5-55b6d0e221e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:27.566394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ded52a-f57a-4d6a-bfd5-55b6d0e221e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:27.567802 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ded52a-f57a-4d6a-bfd5-55b6d0e221e8 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:27.576762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 622/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Fri May 27 19:34:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:27.582600 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68ac45-dfe9-4fef-aba6-5fe1dd24ce54 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:27.669025 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:27.684315 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68ac45-dfe9-4fef-aba6-5fe1dd24ce54 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:27.719952 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5af69-5042-4963-aa52-a577ac529124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:27.720752 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5af69-5042-4963-aa52-a577ac529124 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:27.721073 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5af69-5042-4963-aa52-a577ac529124 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:27.721362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d5af69-5042-4963-aa52-a577ac529124 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:27.734113 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5af69-5042-4963-aa52-a577ac529124 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:27.735613 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d5af69-5042-4963-aa52-a577ac529124 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:27.753638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-c2d5af69-5042-4963-aa52-a577ac529124 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:27.753638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:27.753638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:27.753638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:27.753638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:27.753638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:27.753638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:27.753638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:27.753638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:27.753638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:27.753638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:27.753638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:27.753638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:34:27.753638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:34:27.753638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:34:27.754764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:34:27.754764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:27.754764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:27.754764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:27.754764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:27.754764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:27.754764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:27.754764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:27.754764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:34:27.754764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:27.754764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:27.754764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:27.754764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:27.754764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:27.754764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:27.754764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:27.754764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:27.756054 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:34:27.756054 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:34:27.756054 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:27.756054 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:34:27.756054 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 615/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:34:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:27.761074 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ded52a-f57a-4d6a-bfd5-55b6d0e221e8 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:27.865547 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ded52a-f57a-4d6a-bfd5-55b6d0e221e8 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:27.875022 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:27.901945 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf305ada-65aa-4d18-b706-1efdb1a90ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:27.902623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf305ada-65aa-4d18-b706-1efdb1a90ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:27.902903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf305ada-65aa-4d18-b706-1efdb1a90ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:27.903174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf305ada-65aa-4d18-b706-1efdb1a90ea6 None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:27.903890 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf305ada-65aa-4d18-b706-1efdb1a90ea6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:27.904190 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf305ada-65aa-4d18-b706-1efdb1a90ea6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:27.905609 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf305ada-65aa-4d18-b706-1efdb1a90ea6 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:27.918846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 623/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:27.924458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5af69-5042-4963-aa52-a577ac529124 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:28.030615 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d5af69-5042-4963-aa52-a577ac529124 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:28.069312 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e69fd-4613-4ccb-955c-5f58af4ba586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:28.070126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e69fd-4613-4ccb-955c-5f58af4ba586 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:28.070415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e69fd-4613-4ccb-955c-5f58af4ba586 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:28.070693 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e69fd-4613-4ccb-955c-5f58af4ba586 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:28.073893 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:28.079280 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e69fd-4613-4ccb-955c-5f58af4ba586 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:28.080762 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06e69fd-4613-4ccb-955c-5f58af4ba586 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:28.095775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-d06e69fd-4613-4ccb-955c-5f58af4ba586 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 19:34:28.095775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:28.095775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:28.095775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:28.095775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:28.095775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:28.095775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:28.095775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:28.095775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:28.095775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:28.095775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:28.095775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:28.095775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:34:28.095775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:34:28.095775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:34:28.095775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:34:28.097007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:28.097007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:28.097007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:34:28.097007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:34:28.097007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:34:28.097007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:34:28.097007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:28.097007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:28.097007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:28.097007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:28.097007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:28.097007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:34:28.097007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:28.097007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:28.097007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:28.097007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:28.097007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:28.097007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:28.098357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:28.098357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:28.098357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:34:28.098357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:34:28.098357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:34:28.098357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:34:28.098357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:34:28.098357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:34:28.098357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 19:34:28.098357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:34:28.098357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 616/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:34:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:28.102862 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf305ada-65aa-4d18-b706-1efdb1a90ea6 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:28.206289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf305ada-65aa-4d18-b706-1efdb1a90ea6 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:28.241167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffeffa9-0694-45c8-81e1-e4b55348a40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:28.241843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffeffa9-0694-45c8-81e1-e4b55348a40b None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:28.242128 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffeffa9-0694-45c8-81e1-e4b55348a40b None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:28.242409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffeffa9-0694-45c8-81e1-e4b55348a40b None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:28.243247 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffeffa9-0694-45c8-81e1-e4b55348a40b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:28.243552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffeffa9-0694-45c8-81e1-e4b55348a40b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:28.245664 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffeffa9-0694-45c8-81e1-e4b55348a40b None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:28.267799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 624/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Fri May 27 19:34:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:28.273616 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e69fd-4613-4ccb-955c-5f58af4ba586 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:28.377839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e69fd-4613-4ccb-955c-5f58af4ba586 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:28.414052 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250b7fc-9678-47c1-b81c-51cab35b4f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:28.414705 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250b7fc-9678-47c1-b81c-51cab35b4f46 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:28.414993 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250b7fc-9678-47c1-b81c-51cab35b4f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:28.415274 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3250b7fc-9678-47c1-b81c-51cab35b4f46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:28.480886 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:28.487417 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250b7fc-9678-47c1-b81c-51cab35b4f46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:28.488930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3250b7fc-9678-47c1-b81c-51cab35b4f46 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:28.594578 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 617/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:34:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 19:34:28.600695 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffeffa9-0694-45c8-81e1-e4b55348a40b None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:28.632581 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:28.707543 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffeffa9-0694-45c8-81e1-e4b55348a40b None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:28.744429 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464559d-9429-4d73-933b-956a4c877058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:28.745290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464559d-9429-4d73-933b-956a4c877058 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:28.745489 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464559d-9429-4d73-933b-956a4c877058 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:28.745761 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e464559d-9429-4d73-933b-956a4c877058 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:28.756287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e464559d-9429-4d73-933b-956a4c877058 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:28.758322 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e464559d-9429-4d73-933b-956a4c877058 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:28.777174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-e464559d-9429-4d73-933b-956a4c877058 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:28.777174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:28.777174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:28.777174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:28.777174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:28.777174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:28.777174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:28.777174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:28.777174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:28.777174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:28.777174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:28.777174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:28.777174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:34:28.777174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:34:28.777174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:34:28.782317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:34:28.782317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:28.782317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:28.782317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:28.782317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:28.782317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:28.782317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:28.782317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:28.782317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:34:28.782317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:28.782317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:28.782317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:28.782317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:28.782317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:28.782317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:28.782317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:28.782317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:28.783534 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:34:28.783534 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:34:28.783534 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:28.783534 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:28.783534 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 625/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:34:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:28.784273 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250b7fc-9678-47c1-b81c-51cab35b4f46 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:28.880834 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:28.887547 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3250b7fc-9678-47c1-b81c-51cab35b4f46 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:28.922120 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543e6fc-35e3-461e-8919-ae4cb1c9cfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:28.922815 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543e6fc-35e3-461e-8919-ae4cb1c9cfcf None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:28.923139 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543e6fc-35e3-461e-8919-ae4cb1c9cfcf None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:28.923380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543e6fc-35e3-461e-8919-ae4cb1c9cfcf None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:28.924179 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5543e6fc-35e3-461e-8919-ae4cb1c9cfcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:28.924488 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5543e6fc-35e3-461e-8919-ae4cb1c9cfcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:28.925904 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5543e6fc-35e3-461e-8919-ae4cb1c9cfcf None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:28.937701 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 618/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:34:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:28.943466 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464559d-9429-4d73-933b-956a4c877058 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:29.025230 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:29.048835 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e464559d-9429-4d73-933b-956a4c877058 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:29.086171 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74475f6-d3ce-4cb7-87b5-558ab4ba0975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:29.086919 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74475f6-d3ce-4cb7-87b5-558ab4ba0975 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:29.087205 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74475f6-d3ce-4cb7-87b5-558ab4ba0975 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:29.087484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74475f6-d3ce-4cb7-87b5-558ab4ba0975 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:29.094969 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74475f6-d3ce-4cb7-87b5-558ab4ba0975 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:29.096457 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74475f6-d3ce-4cb7-87b5-558ab4ba0975 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:29.110394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-c74475f6-d3ce-4cb7-87b5-558ab4ba0975 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:34:29.110394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:29.110394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:29.110394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:29.110394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:29.110394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:29.110394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:29.110394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:29.110394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:29.110394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:29.110394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:29.110394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:29.110394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:34:29.110394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:34:29.110394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:34:29.111499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:34:29.111499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:29.111499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:29.111499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:29.111499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:29.111499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:29.111499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:29.111499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:29.111499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:34:29.111499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:29.111499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:29.111499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:29.111499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:29.111499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:29.111499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:29.111499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:29.111499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:29.112665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:34:29.112665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:34:29.112665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:34:29.112665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:34:29.112665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:34:29.112665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:34:29.112665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:34:29.112665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:29.112665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 626/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:34:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:29.117607 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543e6fc-35e3-461e-8919-ae4cb1c9cfcf None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:29.222530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543e6fc-35e3-461e-8919-ae4cb1c9cfcf None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:29.257528 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e1149f-35b8-455d-8708-1ab897508381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:29.258195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1149f-35b8-455d-8708-1ab897508381 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:29.258476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1149f-35b8-455d-8708-1ab897508381 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:29.258751 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e1149f-35b8-455d-8708-1ab897508381 None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:29.259538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e1149f-35b8-455d-8708-1ab897508381 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:29.259849 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e1149f-35b8-455d-8708-1ab897508381 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:29.262044 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e1149f-35b8-455d-8708-1ab897508381 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:29.274362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 619/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:34:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:29.281010 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74475f6-d3ce-4cb7-87b5-558ab4ba0975 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:29.386896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74475f6-d3ce-4cb7-87b5-558ab4ba0975 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:29.422868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedcd2c-f020-4a34-b08f-2e5fcae9fc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:29.423526 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedcd2c-f020-4a34-b08f-2e5fcae9fc95 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:29.423807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedcd2c-f020-4a34-b08f-2e5fcae9fc95 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:29.424084 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedcd2c-f020-4a34-b08f-2e5fcae9fc95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:29.424911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eedcd2c-f020-4a34-b08f-2e5fcae9fc95 None admin] RBAC: Adding query filter params (role.id=11e7404599ae49afb5edd22dabe1a5b2, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=ffc2ed13a0c84f128b795002892efc2f) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:29.425253 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eedcd2c-f020-4a34-b08f-2e5fcae9fc95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:29.426768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eedcd2c-f020-4a34-b08f-2e5fcae9fc95 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:29.436142 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 627/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:34:29 2022] GET /identity/v3/role_assignments?user.id=ffc2ed13a0c84f128b795002892efc2f&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=11e7404599ae49afb5edd22dabe1a5b2 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:31.708834 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f61ff-4606-47a1-9e2d-e8d560ab6589 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:31.709408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f61ff-4606-47a1-9e2d-e8d560ab6589 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:31.709523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922f61ff-4606-47a1-9e2d-e8d560ab6589 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:31.710813 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 620/1247] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:34:31.716615 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2d5f7-63ac-4fc6-92db-3a707e09634e None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:31.717056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2d5f7-63ac-4fc6-92db-3a707e09634e None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:31.717204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c2d5f7-63ac-4fc6-92db-3a707e09634e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:31.722172 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:31.734526 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-54c2d5f7-63ac-4fc6-92db-3a707e09634e None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:34:31.735692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-54c2d5f7-63ac-4fc6-92db-3a707e09634e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:34:31.802468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 628/1248] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:34:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 19:34:31.826590 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f61ff-4606-47a1-9e2d-e8d560ab6589 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:31.845680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:31.927507 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922f61ff-4606-47a1-9e2d-e8d560ab6589 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:31.961919 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba8740-5165-4bf5-a25c-76d8e016f014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:31.962712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba8740-5165-4bf5-a25c-76d8e016f014 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:31.962995 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba8740-5165-4bf5-a25c-76d8e016f014 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:31.963275 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba8740-5165-4bf5-a25c-76d8e016f014 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:31.973643 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba8740-5165-4bf5-a25c-76d8e016f014 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:34:31.981818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-5aba8740-5165-4bf5-a25c-76d8e016f014 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:34:31.981818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:31.981818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:31.981818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:31.981818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:31.981818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:31.981818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:31.981818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:31.981818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:31.981818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:31.981818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:31.981818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:31.981818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:34:31.981818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:34:31.981818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:34:31.981818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:34:31.982840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:34:31.982840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:34:31.982840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:34:31.982840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:34:31.982840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:31.982840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:31.982840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:31.982840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:31.982840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:31.982840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:31.982840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:31.982840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:34:31.982840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:31.982840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:31.982840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:31.982840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:31.983777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:31.983777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:31.983777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:31.983777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:31.983777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:34:31.983777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:34:31.983777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:34:31.983777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:34:31.983777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:34:31.983777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:34:31.983777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:34:31.983777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:34:31.983777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 621/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:34:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:31.989180 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2d5f7-63ac-4fc6-92db-3a707e09634e None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:32.089270 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c2d5f7-63ac-4fc6-92db-3a707e09634e None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:32.121427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:32.126181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08113351-13d7-46e5-8130-13b1e5ec02b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:32.126870 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08113351-13d7-46e5-8130-13b1e5ec02b3 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:32.127152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08113351-13d7-46e5-8130-13b1e5ec02b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:32.127426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08113351-13d7-46e5-8130-13b1e5ec02b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:32.128164 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08113351-13d7-46e5-8130-13b1e5ec02b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:32.128468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08113351-13d7-46e5-8130-13b1e5ec02b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:32.130035 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08113351-13d7-46e5-8130-13b1e5ec02b3 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:32.139043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 629/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:34:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:34:32.144812 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba8740-5165-4bf5-a25c-76d8e016f014 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:32.225092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:32.248265 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba8740-5165-4bf5-a25c-76d8e016f014 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:32.283172 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17db7ab-eebd-464e-a560-9f4e2ccd027e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:32.283993 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17db7ab-eebd-464e-a560-9f4e2ccd027e None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:32.284279 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17db7ab-eebd-464e-a560-9f4e2ccd027e None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:32.284556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17db7ab-eebd-464e-a560-9f4e2ccd027e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:32.295638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17db7ab-eebd-464e-a560-9f4e2ccd027e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:32.297121 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17db7ab-eebd-464e-a560-9f4e2ccd027e None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:32.314706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-f17db7ab-eebd-464e-a560-9f4e2ccd027e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:32.314706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:32.314706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:32.314706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:32.314706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:32.314706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:32.314706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:32.314706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:32.314706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:32.314706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:32.314706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:32.314706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:32.314706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:34:32.314706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:34:32.314706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:34:32.315827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:34:32.315827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:32.315827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:32.315827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:32.315827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:32.315827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:32.315827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:32.315827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:32.315827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:34:32.315827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:32.315827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:32.315827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:32.315827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:32.315827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:32.315827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:32.315827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:32.315827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:32.322103 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:34:32.322103 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:34:32.322103 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:32.322103 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:34:32.322103 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 622/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:34:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:32.322674 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08113351-13d7-46e5-8130-13b1e5ec02b3 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:32.424392 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08113351-13d7-46e5-8130-13b1e5ec02b3 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:32.459483 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd1f0b-69a5-4e88-bcdc-c4f086aa4be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:32.460149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd1f0b-69a5-4e88-bcdc-c4f086aa4be7 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:32.460438 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd1f0b-69a5-4e88-bcdc-c4f086aa4be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:32.460717 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd1f0b-69a5-4e88-bcdc-c4f086aa4be7 None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:32.461473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd1f0b-69a5-4e88-bcdc-c4f086aa4be7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:32.461805 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd1f0b-69a5-4e88-bcdc-c4f086aa4be7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:32.463196 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd1f0b-69a5-4e88-bcdc-c4f086aa4be7 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:32.475309 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 630/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:32.481925 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17db7ab-eebd-464e-a560-9f4e2ccd027e None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:32.610758 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17db7ab-eebd-464e-a560-9f4e2ccd027e None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:32.637362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:32.645102 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4defe1bc-d194-4ade-bdbd-2e35bfb902c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:32.645953 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defe1bc-d194-4ade-bdbd-2e35bfb902c6 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:32.646236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defe1bc-d194-4ade-bdbd-2e35bfb902c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:32.646512 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4defe1bc-d194-4ade-bdbd-2e35bfb902c6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:32.659242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4defe1bc-d194-4ade-bdbd-2e35bfb902c6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:32.660690 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4defe1bc-d194-4ade-bdbd-2e35bfb902c6 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:32.676221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-4defe1bc-d194-4ade-bdbd-2e35bfb902c6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 19:34:32.676221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:32.676221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:32.676221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:32.676221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:32.676221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:32.676221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:32.676221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:32.676221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:32.676221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:32.676221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:32.676221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:32.676221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:34:32.676221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:34:32.676221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:34:32.676221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:34:32.677540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:32.677540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:32.677540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:34:32.677540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:34:32.677540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:34:32.677540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:34:32.677540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:32.677540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:32.677540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:32.677540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:32.677540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:32.677540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:34:32.677540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:32.677540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:32.677540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:32.677540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:32.677540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:32.677540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:32.678702 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:32.678702 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:32.678702 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:34:32.678702 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:34:32.678702 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:34:32.678702 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:34:32.678702 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:34:32.678702 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:34:32.678702 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 19:34:32.678702 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:34:32.678702 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 623/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:34:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:32.683917 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd1f0b-69a5-4e88-bcdc-c4f086aa4be7 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:32.740705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:32.789303 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd1f0b-69a5-4e88-bcdc-c4f086aa4be7 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:32.826320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb4fc2-3717-44e6-aeb7-aa0da88ddb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:32.827305 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb4fc2-3717-44e6-aeb7-aa0da88ddb7a None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:32.827758 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb4fc2-3717-44e6-aeb7-aa0da88ddb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:32.828050 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb4fc2-3717-44e6-aeb7-aa0da88ddb7a None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:32.829121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb4fc2-3717-44e6-aeb7-aa0da88ddb7a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:32.829471 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb4fc2-3717-44e6-aeb7-aa0da88ddb7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:32.831346 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb4fc2-3717-44e6-aeb7-aa0da88ddb7a None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:32.855417 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 631/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Fri May 27 19:34:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:32.861787 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4defe1bc-d194-4ade-bdbd-2e35bfb902c6 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:32.968341 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4defe1bc-d194-4ade-bdbd-2e35bfb902c6 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:33.002037 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e476b32b-fea0-4565-a6ca-77cc1c5848ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:33.002683 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476b32b-fea0-4565-a6ca-77cc1c5848ce None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:33.002962 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476b32b-fea0-4565-a6ca-77cc1c5848ce None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:33.003397 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476b32b-fea0-4565-a6ca-77cc1c5848ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:33.060638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:33.072682 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e476b32b-fea0-4565-a6ca-77cc1c5848ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:33.074208 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e476b32b-fea0-4565-a6ca-77cc1c5848ce None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:33.177555 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 624/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:34:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 19:34:33.183553 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb4fc2-3717-44e6-aeb7-aa0da88ddb7a None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:33.287020 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb4fc2-3717-44e6-aeb7-aa0da88ddb7a None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:33.301712 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:33.322117 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e540568-6381-49a6-ba67-b8bf3457e04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:33.322908 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e540568-6381-49a6-ba67-b8bf3457e04a None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:33.323192 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e540568-6381-49a6-ba67-b8bf3457e04a None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:33.324234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e540568-6381-49a6-ba67-b8bf3457e04a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:33.333976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e540568-6381-49a6-ba67-b8bf3457e04a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:33.335444 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e540568-6381-49a6-ba67-b8bf3457e04a None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:33.353557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-0e540568-6381-49a6-ba67-b8bf3457e04a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:33.353557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:33.353557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:33.353557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:33.353557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:33.353557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:33.353557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:33.353557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:33.353557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:33.353557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:33.353557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:33.353557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:33.353557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:34:33.353557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:34:33.353557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:34:33.358797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:34:33.358797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:33.358797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:33.358797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:33.358797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:33.358797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:33.358797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:33.358797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:33.358797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:34:33.358797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:33.358797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:33.358797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:33.358797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:33.358797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:33.358797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:33.358797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:33.358797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:33.359910 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:34:33.359910 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:34:33.359910 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:33.359910 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:33.359910 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 632/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:34:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:33.361379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e476b32b-fea0-4565-a6ca-77cc1c5848ce None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:33.451899 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:33.464362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e476b32b-fea0-4565-a6ca-77cc1c5848ce None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:33.503886 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2694c1b-512b-4dec-9493-52649831768f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:33.504562 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2694c1b-512b-4dec-9493-52649831768f None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:33.504847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2694c1b-512b-4dec-9493-52649831768f None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:33.505169 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2694c1b-512b-4dec-9493-52649831768f None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:33.505909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2694c1b-512b-4dec-9493-52649831768f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:33.506267 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2694c1b-512b-4dec-9493-52649831768f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:33.507814 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2694c1b-512b-4dec-9493-52649831768f None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:33.521096 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 625/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:33.536556 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e540568-6381-49a6-ba67-b8bf3457e04a None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:33.641635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e540568-6381-49a6-ba67-b8bf3457e04a None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:33.677435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abe756-fdae-4354-910a-1add9a89c958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:33.678276 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abe756-fdae-4354-910a-1add9a89c958 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:33.678561 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abe756-fdae-4354-910a-1add9a89c958 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:33.678865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66abe756-fdae-4354-910a-1add9a89c958 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:33.685893 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66abe756-fdae-4354-910a-1add9a89c958 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:33.687345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66abe756-fdae-4354-910a-1add9a89c958 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:33.701570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-66abe756-fdae-4354-910a-1add9a89c958 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:34:33.701570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:33.701570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:33.701570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:33.701570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:33.701570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:33.701570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:33.701570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:33.701570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:33.701570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:33.701570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:33.701570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:33.701570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:34:33.701570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:34:33.701570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:34:33.702749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:34:33.702749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:33.702749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:33.702749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:33.702749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:33.702749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:33.702749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:33.702749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:33.702749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:34:33.702749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:33.702749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:33.702749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:33.702749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:33.702749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:33.702749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:33.702749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:33.702749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:33.704132 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:34:33.704132 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:34:33.704132 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:34:33.704132 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:34:33.704132 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:34:33.704132 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:34:33.704132 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:34:33.704132 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:33.704132 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 633/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:34:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:33.708883 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2694c1b-512b-4dec-9493-52649831768f None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:33.814461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2694c1b-512b-4dec-9493-52649831768f None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:33.867362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24858213-7885-49e3-bc8a-0113fbe2ac52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:33.868383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24858213-7885-49e3-bc8a-0113fbe2ac52 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:33.868783 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24858213-7885-49e3-bc8a-0113fbe2ac52 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:33.869230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24858213-7885-49e3-bc8a-0113fbe2ac52 None admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:33.870600 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24858213-7885-49e3-bc8a-0113fbe2ac52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:33.871326 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24858213-7885-49e3-bc8a-0113fbe2ac52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:33.873895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24858213-7885-49e3-bc8a-0113fbe2ac52 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:33.893638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 626/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:34:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:33.900136 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abe756-fdae-4354-910a-1add9a89c958 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:33.926980 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:34.001134 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66abe756-fdae-4354-910a-1add9a89c958 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:34.035393 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0e095-d0b1-4600-866d-92fe448d878c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:34.036054 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0e095-d0b1-4600-866d-92fe448d878c None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:34.036337 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0e095-d0b1-4600-866d-92fe448d878c None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:34.036615 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0e095-d0b1-4600-866d-92fe448d878c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:34.037472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0e095-d0b1-4600-866d-92fe448d878c None admin] RBAC: Adding query filter params (role.id=6505bbb129404436966ee59aa7a01d8e, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=ffc2ed13a0c84f128b795002892efc2f) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:34.037787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0e095-d0b1-4600-866d-92fe448d878c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:34.039227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0e095-d0b1-4600-866d-92fe448d878c None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:34.044716 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 634/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:34:33 2022] GET /identity/v3/role_assignments?user.id=ffc2ed13a0c84f128b795002892efc2f&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=6505bbb129404436966ee59aa7a01d8e => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:36.275797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334234a-26f9-4396-8efd-ef864f665fc7 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:36.276372 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334234a-26f9-4396-8efd-ef864f665fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:36.276372 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334234a-26f9-4396-8efd-ef864f665fc7 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:36.277893 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 627/1261] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:34:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:34:36.283321 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddcf2f-0910-48fa-bb92-ea61dcfa74ca None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:36.283753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddcf2f-0910-48fa-bb92-ea61dcfa74ca None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:36.283922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ddcf2f-0910-48fa-bb92-ea61dcfa74ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:36.301432 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-b5ddcf2f-0910-48fa-bb92-ea61dcfa74ca None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:34:36.302673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-b5ddcf2f-0910-48fa-bb92-ea61dcfa74ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:34:36.368977 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 635/1262] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:34:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:34:36.393345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334234a-26f9-4396-8efd-ef864f665fc7 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:36.404586 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:36.498811 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334234a-26f9-4396-8efd-ef864f665fc7 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:36.533955 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb76de3-b19c-41a6-af84-51cc9d8be7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:36.534681 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb76de3-b19c-41a6-af84-51cc9d8be7ff None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:36.534961 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb76de3-b19c-41a6-af84-51cc9d8be7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:36.535232 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb76de3-b19c-41a6-af84-51cc9d8be7ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:36.545842 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb76de3-b19c-41a6-af84-51cc9d8be7ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:34:36.553633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-7cb76de3-b19c-41a6-af84-51cc9d8be7ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:34:36.553633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:36.553633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:36.553633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:36.553633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:36.553633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:36.553633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:36.553633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:36.553633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:36.553633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:36.553633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:36.553633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:36.553633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:34:36.553633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:34:36.553633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:34:36.553633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:34:36.554771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:34:36.554771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:34:36.554771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:34:36.554771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:34:36.554771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:36.554771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:36.554771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:36.554771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:36.554771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:36.554771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:36.554771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:36.554771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:34:36.554771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:36.554771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:36.554771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:36.554771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:36.556089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:36.556089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:36.556089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:36.556089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:36.556089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:34:36.556089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:34:36.556089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:34:36.556089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:34:36.556089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:34:36.556089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:34:36.556089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:34:36.556089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:34:36.556089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 628/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:34:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:36.561035 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ddcf2f-0910-48fa-bb92-ea61dcfa74ca None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:36.579754 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:36.661726 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ddcf2f-0910-48fa-bb92-ea61dcfa74ca None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:36.695820 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267cba0-4f25-4870-99a9-8856be19e35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:36.696492 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267cba0-4f25-4870-99a9-8856be19e35f None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:36.696780 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267cba0-4f25-4870-99a9-8856be19e35f None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:36.697094 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267cba0-4f25-4870-99a9-8856be19e35f None admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:36.697837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0267cba0-4f25-4870-99a9-8856be19e35f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:36.698147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0267cba0-4f25-4870-99a9-8856be19e35f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:36.700536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0267cba0-4f25-4870-99a9-8856be19e35f None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:36.708883 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 636/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:34:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:36.715079 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb76de3-b19c-41a6-af84-51cc9d8be7ff None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:36.782157 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:36.815079 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb76de3-b19c-41a6-af84-51cc9d8be7ff None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:36.849620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178bf8e-f73b-4638-8877-ccaf03635086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:36.850424 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178bf8e-f73b-4638-8877-ccaf03635086 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:36.850712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178bf8e-f73b-4638-8877-ccaf03635086 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:36.850992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178bf8e-f73b-4638-8877-ccaf03635086 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:36.863693 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f178bf8e-f73b-4638-8877-ccaf03635086 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:36.865237 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f178bf8e-f73b-4638-8877-ccaf03635086 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:36.882276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-f178bf8e-f73b-4638-8877-ccaf03635086 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:36.882276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:36.882276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:36.882276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:36.882276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:36.882276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:36.882276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:36.882276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:36.882276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:36.882276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:36.882276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:36.882276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:36.882276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:34:36.882276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:34:36.882276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:34:36.883453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:34:36.883453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:36.883453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:36.883453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:36.883453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:36.883453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:36.883453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:36.883453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:36.883453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:34:36.883453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:36.883453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:36.883453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:36.883453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:36.883453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:36.883453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:36.883453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:36.883453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:36.884593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:34:36.884593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:34:36.884593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:36.884593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:34:36.884593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 629/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:34:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:36.889766 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267cba0-4f25-4870-99a9-8856be19e35f None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:36.970871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:36.989478 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267cba0-4f25-4870-99a9-8856be19e35f None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:37.023614 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3defa00b-19bf-43eb-86e2-9d4b7f0033ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:37.024804 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3defa00b-19bf-43eb-86e2-9d4b7f0033ed None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:37.025116 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3defa00b-19bf-43eb-86e2-9d4b7f0033ed None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:37.025414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3defa00b-19bf-43eb-86e2-9d4b7f0033ed None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:37.026120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3defa00b-19bf-43eb-86e2-9d4b7f0033ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:37.026426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3defa00b-19bf-43eb-86e2-9d4b7f0033ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:37.028646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3defa00b-19bf-43eb-86e2-9d4b7f0033ed None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:37.042081 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 637/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:37.048153 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178bf8e-f73b-4638-8877-ccaf03635086 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:37.148466 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f178bf8e-f73b-4638-8877-ccaf03635086 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:37.170321 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:37.185493 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d2f42-8829-4050-9ae5-a89f859914df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:37.186294 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d2f42-8829-4050-9ae5-a89f859914df None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:37.186575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d2f42-8829-4050-9ae5-a89f859914df None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:37.186846 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d2f42-8829-4050-9ae5-a89f859914df None admin] PATH_INFO: `/v3/users/placement` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:37.195025 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d2f42-8829-4050-9ae5-a89f859914df None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:37.196469 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877d2f42-8829-4050-9ae5-a89f859914df None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:37.211134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-877d2f42-8829-4050-9ae5-a89f859914df None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 19:34:37.211134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:37.211134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:37.211134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:37.211134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:37.211134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:37.211134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:37.211134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:37.211134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:37.211134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:37.211134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:37.211134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:37.211134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:34:37.211134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:34:37.211134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:34:37.211134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:34:37.212345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:37.212345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:37.212345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:34:37.212345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:34:37.212345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:34:37.212345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:34:37.212345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:37.212345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:37.212345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:37.212345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:37.212345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:37.212345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:34:37.212345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:37.212345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:37.212345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:37.212345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:37.212345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:37.212345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:37.213419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:37.213419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:37.213419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:34:37.213419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:34:37.213419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:34:37.213419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:34:37.213419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:34:37.213419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:34:37.213419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 19:34:37.213419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:34:37.213419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 630/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:34:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:37.219249 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3defa00b-19bf-43eb-86e2-9d4b7f0033ed None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:37.319563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3defa00b-19bf-43eb-86e2-9d4b7f0033ed None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:37.353729 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d1cb9-e61a-4cf6-8a83-90e248da4dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:37.354400 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d1cb9-e61a-4cf6-8a83-90e248da4dba None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:37.354690 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d1cb9-e61a-4cf6-8a83-90e248da4dba None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:37.354970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8d1cb9-e61a-4cf6-8a83-90e248da4dba None admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:37.356326 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d1cb9-e61a-4cf6-8a83-90e248da4dba None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:37.356644 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d1cb9-e61a-4cf6-8a83-90e248da4dba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:37.358170 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8d1cb9-e61a-4cf6-8a83-90e248da4dba None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:37.367166 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:37.376030 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 638/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Fri May 27 19:34:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:37.381847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d2f42-8829-4050-9ae5-a89f859914df None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:37.483872 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d2f42-8829-4050-9ae5-a89f859914df None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:37.518286 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d4432-7781-4d34-abce-44fc52cdc0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:37.519085 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d4432-7781-4d34-abce-44fc52cdc0cc None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:37.519374 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d4432-7781-4d34-abce-44fc52cdc0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:37.519652 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4d4432-7781-4d34-abce-44fc52cdc0cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:37.531070 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d4432-7781-4d34-abce-44fc52cdc0cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:37.532520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4d4432-7781-4d34-abce-44fc52cdc0cc None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:37.550530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-3b4d4432-7781-4d34-abce-44fc52cdc0cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:37.550530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:37.550530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:37.550530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:37.550530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:37.550530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:37.550530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:37.550530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:37.550530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:37.550530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:37.550530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:37.550530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:37.550530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:34:37.550530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:34:37.550530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:34:37.551689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:34:37.551689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:37.551689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:37.551689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:37.551689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:37.551689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:37.551689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:37.551689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:37.551689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:34:37.551689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:37.551689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:37.551689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:37.551689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:37.551689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:37.551689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:37.551689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:37.551689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:37.557136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:34:37.557136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:34:37.557136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:37.557136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:34:37.557136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 631/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:34:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:37.557809 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d1cb9-e61a-4cf6-8a83-90e248da4dba None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:37.657282 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8d1cb9-e61a-4cf6-8a83-90e248da4dba None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:37.690645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0b051-1b1e-436c-bbf8-7c3132170019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:37.691306 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0b051-1b1e-436c-bbf8-7c3132170019 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:37.691589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0b051-1b1e-436c-bbf8-7c3132170019 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:37.691869 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c0b051-1b1e-436c-bbf8-7c3132170019 None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:37.692580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c0b051-1b1e-436c-bbf8-7c3132170019 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:37.692881 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c0b051-1b1e-436c-bbf8-7c3132170019 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:37.694320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c0b051-1b1e-436c-bbf8-7c3132170019 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:37.706070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 639/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:34:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:37.711734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d4432-7781-4d34-abce-44fc52cdc0cc None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:37.743304 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:37.812613 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4d4432-7781-4d34-abce-44fc52cdc0cc None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:37.849046 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61907c97-6068-456d-9607-212566147a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:37.849851 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61907c97-6068-456d-9607-212566147a97 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:37.850127 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61907c97-6068-456d-9607-212566147a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:37.850401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61907c97-6068-456d-9607-212566147a97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:37.858485 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61907c97-6068-456d-9607-212566147a97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:37.859929 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61907c97-6068-456d-9607-212566147a97 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:37.874470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-61907c97-6068-456d-9607-212566147a97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:34:37.874470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:37.874470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:37.874470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:37.874470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:37.874470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:37.874470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:37.874470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:37.874470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:37.874470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:37.874470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:37.874470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:37.874470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:34:37.874470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:34:37.874470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:34:37.875713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:34:37.875713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:37.875713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:37.875713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:37.875713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:37.875713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:37.875713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:37.875713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:37.875713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:34:37.875713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:37.875713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:37.875713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:37.875713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:37.875713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:37.875713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:37.875713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:37.875713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:37.877300 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:34:37.877300 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:34:37.877300 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:34:37.877300 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:34:37.877300 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:34:37.877300 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:34:37.877300 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:34:37.877300 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:34:37.877300 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 632/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:34:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:37.881609 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0b051-1b1e-436c-bbf8-7c3132170019 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:37.941220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:37.990606 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c0b051-1b1e-436c-bbf8-7c3132170019 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:38.039690 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c967a9e-f7b4-46f6-9d85-7b9fa917e5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:38.039690 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c967a9e-f7b4-46f6-9d85-7b9fa917e5a2 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:38.039690 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c967a9e-f7b4-46f6-9d85-7b9fa917e5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:38.039690 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c967a9e-f7b4-46f6-9d85-7b9fa917e5a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:38.040169 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c967a9e-f7b4-46f6-9d85-7b9fa917e5a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:38.040169 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c967a9e-f7b4-46f6-9d85-7b9fa917e5a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:38.040169 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c967a9e-f7b4-46f6-9d85-7b9fa917e5a2 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:38.041792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 640/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:34:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:38.048184 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61907c97-6068-456d-9607-212566147a97 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:38.142474 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:38.157228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61907c97-6068-456d-9607-212566147a97 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:38.192152 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4bdae1-dec6-461b-9a6e-ad3efc2fe43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:38.193079 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4bdae1-dec6-461b-9a6e-ad3efc2fe43f None admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:38.193366 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4bdae1-dec6-461b-9a6e-ad3efc2fe43f None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:38.193632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4bdae1-dec6-461b-9a6e-ad3efc2fe43f None admin] PATH_INFO: `/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/ffc2ed13a0c84f128b795002892efc2f/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:38.195564 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4bdae1-dec6-461b-9a6e-ad3efc2fe43f None admin] RBAC: Authorizing `identity:create_grant(project_id=a47d6b6971c3452e9799e8146856bdc5, user_id=ffc2ed13a0c84f128b795002892efc2f, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:38.197244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4bdae1-dec6-461b-9a6e-ad3efc2fe43f None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:38.303497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 633/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Fri May 27 19:34:38 2022] PUT /identity/v3/projects/a47d6b6971c3452e9799e8146856bdc5/users/ffc2ed13a0c84f128b795002892efc2f/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:34:40.534524 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ad068-649e-463b-94e4-485548e6c2eb None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:40.535041 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ad068-649e-463b-94e4-485548e6c2eb None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:40.535135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1ad068-649e-463b-94e4-485548e6c2eb None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:40.536453 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 641/1274] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:34:40.541830 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72693a0a-d950-4c4e-b1e6-9ddfbc3b03a6 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:40.542228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72693a0a-d950-4c4e-b1e6-9ddfbc3b03a6 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:40.542394 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72693a0a-d950-4c4e-b1e6-9ddfbc3b03a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:40.559825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-72693a0a-d950-4c4e-b1e6-9ddfbc3b03a6 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:34:40.566306 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-72693a0a-d950-4c4e-b1e6-9ddfbc3b03a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:34:40.630030 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 634/1275] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:34:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:34:40.654705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ad068-649e-463b-94e4-485548e6c2eb None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:40.755817 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1ad068-649e-463b-94e4-485548e6c2eb None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:40.773469 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:40.790146 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96835c67-9f8a-42bb-b975-e7af45f18703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:40.790934 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96835c67-9f8a-42bb-b975-e7af45f18703 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:40.791228 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96835c67-9f8a-42bb-b975-e7af45f18703 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:40.791506 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96835c67-9f8a-42bb-b975-e7af45f18703 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:40.801505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96835c67-9f8a-42bb-b975-e7af45f18703 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 19:34:40.809106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-96835c67-9f8a-42bb-b975-e7af45f18703 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:34:40.809106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:40.809106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:40.809106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:40.809106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:40.809106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:40.809106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:40.809106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:40.809106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:40.809106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:40.809106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:40.809106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:40.809106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 19:34:40.809106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 19:34:40.809106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 19:34:40.809106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 19:34:40.810813 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 19:34:40.810813 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 19:34:40.810813 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 19:34:40.810813 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 19:34:40.810813 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:40.810813 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:40.810813 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:40.810813 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:40.810813 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:40.810813 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:40.810813 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:40.810813 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:34:40.810813 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:40.810813 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:40.810813 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:40.810813 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:40.811856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:40.811856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:40.811856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:40.811856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:40.811856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 19:34:40.811856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 19:34:40.811856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 19:34:40.811856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 19:34:40.811856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 19:34:40.811856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 19:34:40.811856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 19:34:40.811856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:40.811856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 642/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Fri May 27 19:34:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:40.816367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72693a0a-d950-4c4e-b1e6-9ddfbc3b03a6 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:40.896923 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:40.914751 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72693a0a-d950-4c4e-b1e6-9ddfbc3b03a6 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:40.948184 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc71a64-d957-488f-a9f1-83d1f18c0ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:40.949124 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc71a64-d957-488f-a9f1-83d1f18c0ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:40.950168 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc71a64-d957-488f-a9f1-83d1f18c0ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:40.950363 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc71a64-d957-488f-a9f1-83d1f18c0ea0 None admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:40.951164 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc71a64-d957-488f-a9f1-83d1f18c0ea0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:40.951798 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc71a64-d957-488f-a9f1-83d1f18c0ea0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:40.953365 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc71a64-d957-488f-a9f1-83d1f18c0ea0 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:40.961247 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 635/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Fri May 27 19:34:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:40.967633 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96835c67-9f8a-42bb-b975-e7af45f18703 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:41.067214 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96835c67-9f8a-42bb-b975-e7af45f18703 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:41.103251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f88ec6-a691-40cd-87db-765de8a1050a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:41.104382 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f88ec6-a691-40cd-87db-765de8a1050a None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:41.104382 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f88ec6-a691-40cd-87db-765de8a1050a None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:41.104587 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f88ec6-a691-40cd-87db-765de8a1050a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:41.116982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f88ec6-a691-40cd-87db-765de8a1050a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:41.118462 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f88ec6-a691-40cd-87db-765de8a1050a None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:41.135986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-48f88ec6-a691-40cd-87db-765de8a1050a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:41.135986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:41.135986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:41.135986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:41.135986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:41.135986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:41.135986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:41.135986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:41.135986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:41.135986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:41.135986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:41.135986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:41.135986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:34:41.135986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:34:41.135986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:34:41.137060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:34:41.137060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:41.137060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:41.137060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:41.137060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:41.137060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:41.137060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:41.137060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:41.137060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:34:41.137060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:41.137060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:41.137060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:41.137060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:41.137060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:41.137060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:41.137060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:41.137060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:41.138154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:34:41.138154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:34:41.138154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:41.138154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:41.138154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 643/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:34:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:41.143402 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc71a64-d957-488f-a9f1-83d1f18c0ea0 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:41.245940 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc71a64-d957-488f-a9f1-83d1f18c0ea0 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:41.281019 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7afc9b-3b74-4694-bbea-095e7e9c9876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:41.281808 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7afc9b-3b74-4694-bbea-095e7e9c9876 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:41.282093 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7afc9b-3b74-4694-bbea-095e7e9c9876 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:41.282370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7afc9b-3b74-4694-bbea-095e7e9c9876 None admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:41.283098 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7afc9b-3b74-4694-bbea-095e7e9c9876 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:41.283404 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7afc9b-3b74-4694-bbea-095e7e9c9876 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:41.284778 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7afc9b-3b74-4694-bbea-095e7e9c9876 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:41.297451 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 636/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:41.303412 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f88ec6-a691-40cd-87db-765de8a1050a None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:41.335460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:41.408585 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f88ec6-a691-40cd-87db-765de8a1050a None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:41.442903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831abbb-c064-48e4-8883-f75af1a06ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:41.443726 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831abbb-c064-48e4-8883-f75af1a06ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:41.444013 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831abbb-c064-48e4-8883-f75af1a06ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:41.444292 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8831abbb-c064-48e4-8883-f75af1a06ff8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:41.452490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8831abbb-c064-48e4-8883-f75af1a06ff8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:41.453994 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8831abbb-c064-48e4-8883-f75af1a06ff8 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:41.469477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-8831abbb-c064-48e4-8883-f75af1a06ff8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 19:34:41.469477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:41.469477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:41.469477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:41.469477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:41.469477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:41.469477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:41.469477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:41.469477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:41.469477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:41.469477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:41.469477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:41.469477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 19:34:41.469477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 19:34:41.469477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 19:34:41.469477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 19:34:41.470656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:41.470656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:41.470656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 19:34:41.470656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:34:41.470656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 19:34:41.470656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 19:34:41.470656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:41.470656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:41.470656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:41.470656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:41.470656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:41.470656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:34:41.470656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:41.470656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:41.470656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:41.470656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:41.470656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:41.470656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:41.471865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:41.471865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:41.471865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 19:34:41.471865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 19:34:41.471865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 19:34:41.471865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 19:34:41.471865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 19:34:41.471865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 19:34:41.471865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 19:34:41.471865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:41.471865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 644/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:34:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:41.476663 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7afc9b-3b74-4694-bbea-095e7e9c9876 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:41.481205 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:41.583392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7afc9b-3b74-4694-bbea-095e7e9c9876 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:41.619577 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40d5d9-5fb5-4403-a095-072a23fb19e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:41.620185 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40d5d9-5fb5-4403-a095-072a23fb19e3 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:41.620471 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40d5d9-5fb5-4403-a095-072a23fb19e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:41.620750 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40d5d9-5fb5-4403-a095-072a23fb19e3 None admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:41.622017 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40d5d9-5fb5-4403-a095-072a23fb19e3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:41.622330 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40d5d9-5fb5-4403-a095-072a23fb19e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:41.623779 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee40d5d9-5fb5-4403-a095-072a23fb19e3 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:41.642899 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 637/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Fri May 27 19:34:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:41.648812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831abbb-c064-48e4-8883-f75af1a06ff8 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:41.740505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:41.751329 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8831abbb-c064-48e4-8883-f75af1a06ff8 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:41.786369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361a395-4477-4809-8ea3-2f7590931519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:41.787036 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361a395-4477-4809-8ea3-2f7590931519 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:41.787323 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361a395-4477-4809-8ea3-2f7590931519 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:41.787606 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f361a395-4477-4809-8ea3-2f7590931519 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:41.857472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361a395-4477-4809-8ea3-2f7590931519 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:41.858940 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f361a395-4477-4809-8ea3-2f7590931519 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:41.952659 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:41.963184 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 645/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Fri May 27 19:34:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 19:34:41.969697 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40d5d9-5fb5-4403-a095-072a23fb19e3 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:42.037561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:42.077760 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee40d5d9-5fb5-4403-a095-072a23fb19e3 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:42.112916 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e035d13-4777-4162-a04b-04164505e871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:42.113793 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e035d13-4777-4162-a04b-04164505e871 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:42.114081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e035d13-4777-4162-a04b-04164505e871 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:42.114362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e035d13-4777-4162-a04b-04164505e871 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:42.126694 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e035d13-4777-4162-a04b-04164505e871 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:42.128171 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e035d13-4777-4162-a04b-04164505e871 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:42.148523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-3e035d13-4777-4162-a04b-04164505e871 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:42.148523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:42.148523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:42.148523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:42.148523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:42.148523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:42.148523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:42.148523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:42.148523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:42.148523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:42.148523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:42.148523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:42.148523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 19:34:42.148523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 19:34:42.148523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 19:34:42.149732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 19:34:42.149732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:42.149732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:42.149732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:42.149732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:42.149732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:42.149732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:42.149732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:42.149732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:34:42.149732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:42.149732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:42.149732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:42.149732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:42.149732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:42.149732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:42.149732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:42.149732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:42.150858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 19:34:42.150858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 19:34:42.150858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 19:34:42.150858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:34:42.150858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 638/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:34:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:42.156074 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361a395-4477-4809-8ea3-2f7590931519 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:42.262863 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f361a395-4477-4809-8ea3-2f7590931519 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:42.298942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e2768-e874-4d63-bb71-864734e6ef5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:42.299622 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e2768-e874-4d63-bb71-864734e6ef5a None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:42.299910 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e2768-e874-4d63-bb71-864734e6ef5a None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:42.300188 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e2768-e874-4d63-bb71-864734e6ef5a None admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:42.300919 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e2768-e874-4d63-bb71-864734e6ef5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:42.301266 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e2768-e874-4d63-bb71-864734e6ef5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:42.302681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e2768-e874-4d63-bb71-864734e6ef5a None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:42.315098 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 646/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Fri May 27 19:34:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:42.321026 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e035d13-4777-4162-a04b-04164505e871 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:42.430507 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e035d13-4777-4162-a04b-04164505e871 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:42.447835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:42.470978 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2ef90-fbfc-45d9-a121-a0f55434009f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:42.471790 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2ef90-fbfc-45d9-a121-a0f55434009f None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:42.472079 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2ef90-fbfc-45d9-a121-a0f55434009f None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:42.472369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2ef90-fbfc-45d9-a121-a0f55434009f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:42.479812 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2ef90-fbfc-45d9-a121-a0f55434009f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:42.485128 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2ef90-fbfc-45d9-a121-a0f55434009f None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:42.497518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application [None req-90f2ef90-fbfc-45d9-a121-a0f55434009f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:34:42.497518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:42.497518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:42.497518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:42.497518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:42.497518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:42.497518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:42.497518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:42.497518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:42.497518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:42.497518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:42.497518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:42.497518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 19:34:42.497518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 19:34:42.497518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 19:34:42.498685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 19:34:42.498685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:42.498685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:42.498685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:42.498685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:42.498685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:42.498685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:42.498685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:42.498685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application with Lock( May 27 19:34:42.498685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:42.498685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:42.498685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:42.498685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:42.498685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:42.498685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:42.498685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:42.498685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:42.499698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 19:34:42.499698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 19:34:42.499698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 19:34:42.499698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 19:34:42.499698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 19:34:42.499698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 19:34:42.499698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 19:34:42.499698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: ERROR keystone.server.flask.application May 27 19:34:42.499698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 639/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Fri May 27 19:34:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 19:34:42.505052 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e2768-e874-4d63-bb71-864734e6ef5a None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:42.577853 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:42.656589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e2768-e874-4d63-bb71-864734e6ef5a None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:42.691453 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8f204-e236-47a2-b28a-4b659b07154b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:42.692161 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8f204-e236-47a2-b28a-4b659b07154b None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:42.692482 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8f204-e236-47a2-b28a-4b659b07154b None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:42.692761 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8f204-e236-47a2-b28a-4b659b07154b None admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:42.693582 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8f204-e236-47a2-b28a-4b659b07154b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:42.693895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8f204-e236-47a2-b28a-4b659b07154b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:42.695351 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8f204-e236-47a2-b28a-4b659b07154b None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:42.709945 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 647/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Fri May 27 19:34:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:42.716998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2ef90-fbfc-45d9-a121-a0f55434009f None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:42.830578 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2ef90-fbfc-45d9-a121-a0f55434009f None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:42.871662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56701341-656b-49bc-acff-ac557d546e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:42.872395 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56701341-656b-49bc-acff-ac557d546e57 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:42.872679 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56701341-656b-49bc-acff-ac557d546e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:42.872981 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56701341-656b-49bc-acff-ac557d546e57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:42.874036 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56701341-656b-49bc-acff-ac557d546e57 None admin] RBAC: Adding query filter params (role.id=6505bbb129404436966ee59aa7a01d8e, scope.project.id=a47d6b6971c3452e9799e8146856bdc5, user.id=ffc2ed13a0c84f128b795002892efc2f) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:42.874348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56701341-656b-49bc-acff-ac557d546e57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:42.878832 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56701341-656b-49bc-acff-ac557d546e57 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:42.882267 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 640/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Fri May 27 19:34:42 2022] GET /identity/v3/role_assignments?user.id=ffc2ed13a0c84f128b795002892efc2f&scope.project.id=a47d6b6971c3452e9799e8146856bdc5&role.id=6505bbb129404436966ee59aa7a01d8e => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:45.161634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafb8e7-da52-4823-b208-dd58e775e3ad None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:45.162173 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafb8e7-da52-4823-b208-dd58e775e3ad None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:45.162300 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafb8e7-da52-4823-b208-dd58e775e3ad None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:45.163589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 648/1288] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 19:34:45.169591 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a7b59-824b-4e0e-baca-300bc6e90edb None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:45.170056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a7b59-824b-4e0e-baca-300bc6e90edb None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:45.170150 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a7b59-824b-4e0e-baca-300bc6e90edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:45.187292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-a87a7b59-824b-4e0e-baca-300bc6e90edb None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:34:45.188478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-a87a7b59-824b-4e0e-baca-300bc6e90edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:34:45.218249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:45.256685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 641/1289] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:34:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:34:45.278834 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafb8e7-da52-4823-b208-dd58e775e3ad None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:45.382203 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafb8e7-da52-4823-b208-dd58e775e3ad None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:45.393073 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:45.416742 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d2a12-c944-4a66-953a-376b6ca688ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:45.417537 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d2a12-c944-4a66-953a-376b6ca688ae None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:45.417819 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d2a12-c944-4a66-953a-376b6ca688ae None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:45.418092 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d2a12-c944-4a66-953a-376b6ca688ae None admin] PATH_INFO: `/v3/services/placement` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:45.419529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208d2a12-c944-4a66-953a-376b6ca688ae None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:45.420887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208d2a12-c944-4a66-953a-376b6ca688ae None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:45.432566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-208d2a12-c944-4a66-953a-376b6ca688ae None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 19:34:45.432566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:45.432566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:34:45.432566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:34:45.432566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:34:45.432566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:34:45.432566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:34:45.432566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:34:45.432566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 19:34:45.432566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:45.432566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:34:45.432566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:45.432566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:45.432566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:45.432566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:45.432566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:45.432566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:45.433832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:45.433832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:45.433832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:45.433832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:45.433832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:45.433832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:45.433832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:34:45.433832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:34:45.433832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:34:45.433832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:34:45.433832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:45.433832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:45.433832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:45.433832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:45.433832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:45.433832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:45.436076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:45.436076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:34:45.436076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:45.436076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:45.436076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:45.436076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:45.436076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:45.436076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:45.436076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:45.436076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:45.436076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:34:45.436076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:34:45.436076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 19:34:45.436076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:45.436076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 649/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Fri May 27 19:34:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:45.440769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a7b59-824b-4e0e-baca-300bc6e90edb None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:45.540753 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a7b59-824b-4e0e-baca-300bc6e90edb None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:45.574361 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48da01e4-22be-4cde-aa67-bbeeae46bcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:45.575062 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da01e4-22be-4cde-aa67-bbeeae46bcbd None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:45.575347 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da01e4-22be-4cde-aa67-bbeeae46bcbd None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:45.575629 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da01e4-22be-4cde-aa67-bbeeae46bcbd None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:45.576376 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48da01e4-22be-4cde-aa67-bbeeae46bcbd None admin] RBAC: Adding query filter params (name=placement) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:45.576683 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48da01e4-22be-4cde-aa67-bbeeae46bcbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:45.578086 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48da01e4-22be-4cde-aa67-bbeeae46bcbd None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:45.582924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 642/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Fri May 27 19:34:45 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:45.588694 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d2a12-c944-4a66-953a-376b6ca688ae None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:45.690424 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d2a12-c944-4a66-953a-376b6ca688ae None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:45.723240 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87537020-9083-4ac2-bf69-0b67f93fdc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:45.723895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87537020-9083-4ac2-bf69-0b67f93fdc52 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:45.724175 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87537020-9083-4ac2-bf69-0b67f93fdc52 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:45.724452 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87537020-9083-4ac2-bf69-0b67f93fdc52 None admin] PATH_INFO: `/v3/services` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:45.725185 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87537020-9083-4ac2-bf69-0b67f93fdc52 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:45.725540 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87537020-9083-4ac2-bf69-0b67f93fdc52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:45.726903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87537020-9083-4ac2-bf69-0b67f93fdc52 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:45.732218 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 650/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Fri May 27 19:34:45 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:47.911316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7223bd-8659-47f0-8403-be7123fd1d70 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:47.911863 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7223bd-8659-47f0-8403-be7123fd1d70 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:47.911863 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7223bd-8659-47f0-8403-be7123fd1d70 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:47.913206 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 643/1293] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:34:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:34:47.919151 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993b9ba-7a79-4160-82b6-6a9b38af9a81 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:47.919932 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993b9ba-7a79-4160-82b6-6a9b38af9a81 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:47.920225 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993b9ba-7a79-4160-82b6-6a9b38af9a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:47.937472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-8993b9ba-7a79-4160-82b6-6a9b38af9a81 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:34:47.938673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-8993b9ba-7a79-4160-82b6-6a9b38af9a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:34:47.969556 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:48.007046 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 651/1294] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:34:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:34:48.030616 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7223bd-8659-47f0-8403-be7123fd1d70 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:48.063759 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:48.134248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7223bd-8659-47f0-8403-be7123fd1d70 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:48.169027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90482627-1a83-4046-8a5d-66ecfdfa0f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:48.169618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90482627-1a83-4046-8a5d-66ecfdfa0f00 None admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:48.169897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90482627-1a83-4046-8a5d-66ecfdfa0f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:48.170177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90482627-1a83-4046-8a5d-66ecfdfa0f00 None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:48.170986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90482627-1a83-4046-8a5d-66ecfdfa0f00 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:48.172707 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90482627-1a83-4046-8a5d-66ecfdfa0f00 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:48.258449 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 644/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Fri May 27 19:34:48 2022] POST /identity/v3/services => generated 246 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:34:50.490518 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546148a4-be9e-4890-adf5-e50822a2fb5f None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:50.491084 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546148a4-be9e-4890-adf5-e50822a2fb5f None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:50.491084 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546148a4-be9e-4890-adf5-e50822a2fb5f None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:50.492481 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 652/1296] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:34:50.498568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac39ab-9338-43d8-842d-2d987a50c9f9 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:50.499048 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac39ab-9338-43d8-842d-2d987a50c9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:50.499192 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac39ab-9338-43d8-842d-2d987a50c9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:50.516559 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-42ac39ab-9338-43d8-842d-2d987a50c9f9 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:34:50.517772 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-42ac39ab-9338-43d8-842d-2d987a50c9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:34:50.592011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 645/1297] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:34:50 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 19:34:50.615537 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546148a4-be9e-4890-adf5-e50822a2fb5f None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:50.718272 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546148a4-be9e-4890-adf5-e50822a2fb5f None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:50.752602 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554d999-7355-488a-81f1-ffa7a298d019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:50.753362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554d999-7355-488a-81f1-ffa7a298d019 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:50.753650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554d999-7355-488a-81f1-ffa7a298d019 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:50.753989 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554d999-7355-488a-81f1-ffa7a298d019 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:50.754639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554d999-7355-488a-81f1-ffa7a298d019 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:50.756039 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554d999-7355-488a-81f1-ffa7a298d019 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:50.768669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-b554d999-7355-488a-81f1-ffa7a298d019 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 19:34:50.768669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:50.768669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:34:50.768669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:34:50.768669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:34:50.768669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:34:50.768669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:34:50.768669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:34:50.768669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 19:34:50.768669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:50.768669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:34:50.768669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:50.768669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:50.768669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:50.768669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:50.768669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:50.768669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:50.769774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:50.769774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:50.769774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:50.769774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:50.769774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:50.769774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:50.769774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:34:50.769774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:34:50.769774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:34:50.769774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:34:50.769774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:50.769774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:50.769774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:50.769774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:50.769774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:50.769774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:50.770645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:50.770645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:34:50.770645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:50.770645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:50.770645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:50.770645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:50.770645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:50.770645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:50.770645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:50.770645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:50.770645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:34:50.770645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:34:50.770645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 19:34:50.770645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:50.770645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 653/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Fri May 27 19:34:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:50.776026 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac39ab-9338-43d8-842d-2d987a50c9f9 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:50.794944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:50.874178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac39ab-9338-43d8-842d-2d987a50c9f9 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:50.908282 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf93c4-2bcc-4a87-b7f2-042f70d4dee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:50.909033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf93c4-2bcc-4a87-b7f2-042f70d4dee5 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:50.909318 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf93c4-2bcc-4a87-b7f2-042f70d4dee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:50.909592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf93c4-2bcc-4a87-b7f2-042f70d4dee5 None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:50.910299 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf93c4-2bcc-4a87-b7f2-042f70d4dee5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:50.910602 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf93c4-2bcc-4a87-b7f2-042f70d4dee5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:50.911966 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf93c4-2bcc-4a87-b7f2-042f70d4dee5 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:50.917062 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 646/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Fri May 27 19:34:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:50.923444 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554d999-7355-488a-81f1-ffa7a298d019 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:50.955781 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:51.023055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554d999-7355-488a-81f1-ffa7a298d019 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:51.057234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529c72b-3ea3-424a-8e5b-eef0bb2746bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:51.057928 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529c72b-3ea3-424a-8e5b-eef0bb2746bc None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:51.058219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529c72b-3ea3-424a-8e5b-eef0bb2746bc None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:51.058486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529c72b-3ea3-424a-8e5b-eef0bb2746bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:51.059823 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529c72b-3ea3-424a-8e5b-eef0bb2746bc None admin] RBAC: Adding query filter params (interface=public, service_id=86dfcb9905104204b0911544c1bd742d, region_id=RegionOne) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:51.060134 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529c72b-3ea3-424a-8e5b-eef0bb2746bc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:51.061516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529c72b-3ea3-424a-8e5b-eef0bb2746bc None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:51.066575 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 654/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Fri May 27 19:34:50 2022] GET /identity/v3/endpoints?service_id=86dfcb9905104204b0911544c1bd742d&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:51.072150 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf93c4-2bcc-4a87-b7f2-042f70d4dee5 None admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:51.153430 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:51.171649 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf93c4-2bcc-4a87-b7f2-042f70d4dee5 None admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:51.205409 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a15e22-108a-40a7-b056-b603097a89be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:51.206014 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a15e22-108a-40a7-b056-b603097a89be None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:51.206300 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a15e22-108a-40a7-b056-b603097a89be None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:51.206578 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a15e22-108a-40a7-b056-b603097a89be None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:51.207259 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a15e22-108a-40a7-b056-b603097a89be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:51.208646 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a15e22-108a-40a7-b056-b603097a89be None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:51.215751 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 647/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Fri May 27 19:34:51 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:34:53.405303 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf7a56-0772-40f9-ac1b-f2eb0d6a2350 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:53.405871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf7a56-0772-40f9-ac1b-f2eb0d6a2350 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:53.405871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cf7a56-0772-40f9-ac1b-f2eb0d6a2350 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:53.407280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 655/1302] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:34:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:34:53.412503 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88fd9f-a9a2-4279-9f22-95767cf7118c None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:53.412902 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88fd9f-a9a2-4279-9f22-95767cf7118c None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:53.413288 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb88fd9f-a9a2-4279-9f22-95767cf7118c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:53.430450 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-fb88fd9f-a9a2-4279-9f22-95767cf7118c None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:34:53.431634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-fb88fd9f-a9a2-4279-9f22-95767cf7118c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:34:53.500051 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 648/1303] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:34:53 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 19:34:53.524329 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf7a56-0772-40f9-ac1b-f2eb0d6a2350 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:53.621455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:53.629270 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cf7a56-0772-40f9-ac1b-f2eb0d6a2350 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:53.663520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2d142-8a2a-4253-8cb8-64fcd891af22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:53.664255 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2d142-8a2a-4253-8cb8-64fcd891af22 None admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:53.664543 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2d142-8a2a-4253-8cb8-64fcd891af22 None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:53.664852 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2d142-8a2a-4253-8cb8-64fcd891af22 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:53.665533 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2d142-8a2a-4253-8cb8-64fcd891af22 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:53.666910 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2d142-8a2a-4253-8cb8-64fcd891af22 None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:53.679591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application [None req-20c2d142-8a2a-4253-8cb8-64fcd891af22 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 19:34:53.679591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:53.679591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 19:34:53.679591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 19:34:53.679591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 19:34:53.679591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 19:34:53.679591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 19:34:53.679591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:34:53.679591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 19:34:53.679591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:53.679591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 19:34:53.679591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:53.679591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 19:34:53.679591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 19:34:53.679591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 19:34:53.679591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 19:34:53.679591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 19:34:53.680611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 19:34:53.680611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 19:34:53.680611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 19:34:53.680611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 19:34:53.680611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 19:34:53.680611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 19:34:53.680611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 19:34:53.680611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 19:34:53.680611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 19:34:53.680611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 19:34:53.680611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 19:34:53.680611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 19:34:53.680611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 19:34:53.680611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 19:34:53.680611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 19:34:53.680611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.get_or_create( May 27 19:34:53.685307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 19:34:53.685307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application with Lock( May 27 19:34:53.685307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 19:34:53.685307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self._enter() May 27 19:34:53.685307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 19:34:53.685307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 19:34:53.685307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 19:34:53.685307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application return self.creator() May 27 19:34:53.685307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 19:34:53.685307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application created_value = creator( May 27 19:34:53.685307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 19:34:53.685307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 19:34:53.685307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 19:34:53.685307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: ERROR keystone.server.flask.application May 27 19:34:53.685307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 656/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Fri May 27 19:34:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 19:34:53.688620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88fd9f-a9a2-4279-9f22-95767cf7118c None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:53.764154 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:34:53.786597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb88fd9f-a9a2-4279-9f22-95767cf7118c None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:53.821819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d326c81f-b6d3-4fe2-aac7-d5f3997c2492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:53.822509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326c81f-b6d3-4fe2-aac7-d5f3997c2492 None admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:53.822798 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326c81f-b6d3-4fe2-aac7-d5f3997c2492 None admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:53.823079 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d326c81f-b6d3-4fe2-aac7-d5f3997c2492 None admin] PATH_INFO: `/v3/services` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:53.824125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d326c81f-b6d3-4fe2-aac7-d5f3997c2492 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:34:53.824408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d326c81f-b6d3-4fe2-aac7-d5f3997c2492 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:53.827249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d326c81f-b6d3-4fe2-aac7-d5f3997c2492 None admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:53.834510 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 649/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Fri May 27 19:34:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:34:53.841423 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2d142-8a2a-4253-8cb8-64fcd891af22 None admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:34:53.940819 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2d142-8a2a-4253-8cb8-64fcd891af22 None admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:34:53.974571 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76908495-2c9f-4fe4-97bf-ba35a143ce4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:34:53.975224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76908495-2c9f-4fe4-97bf-ba35a143ce4b None admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:34:53.975505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76908495-2c9f-4fe4-97bf-ba35a143ce4b None admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:34:53.975808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76908495-2c9f-4fe4-97bf-ba35a143ce4b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:34:53.976611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76908495-2c9f-4fe4-97bf-ba35a143ce4b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:34:53.978017 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76908495-2c9f-4fe4-97bf-ba35a143ce4b None admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:34:54.070715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 657/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Fri May 27 19:34:53 2022] POST /identity/v3/endpoints => generated 331 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:35:03.831481 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=30 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:35:04.066039 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:40:58.792032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a038531-9ab8-4e84-a7c8-db88631acfb9 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:40:58.792616 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a038531-9ab8-4e84-a7c8-db88631acfb9 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:40:58.792616 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a038531-9ab8-4e84-a7c8-db88631acfb9 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:40:58.794007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 650/1307] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:40:58.799397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9168bea-1985-445e-9f14-455e40abefbe None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:40:58.800574 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9168bea-1985-445e-9f14-455e40abefbe None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:40:58.800574 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9168bea-1985-445e-9f14-455e40abefbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:40:58.881543 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-f9168bea-1985-445e-9f14-455e40abefbe None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:40:58.883174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-f9168bea-1985-445e-9f14-455e40abefbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:40:59.045227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 658/1308] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:40:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:40:59.106922 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b1d8d-fba7-4c2b-9d9a-a4dfc959cd55 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:40:59.107218 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b1d8d-fba7-4c2b-9d9a-a4dfc959cd55 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:40:59.107686 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b1d8d-fba7-4c2b-9d9a-a4dfc959cd55 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:40:59.109110 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 651/1309] 10.222.0.13 () {54 vars in 964 bytes} [Fri May 27 19:40:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:40:59.116568 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de880155-93e9-4234-b21d-0cda1266e78b None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:40:59.116818 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de880155-93e9-4234-b21d-0cda1266e78b None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:40:59.117204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de880155-93e9-4234-b21d-0cda1266e78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:40:59.212984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-de880155-93e9-4234-b21d-0cda1266e78b None None] MFA Rules not processed for user `dd34f15400c4476bbb2161dbe1b8b2b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:40:59.214112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-de880155-93e9-4234-b21d-0cda1266e78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:40:59.236006 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:40:59.303519 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 659/1310] 10.222.0.13 () {58 vars in 1061 bytes} [Fri May 27 19:40:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:40:59.314106 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b1d8d-fba7-4c2b-9d9a-a4dfc959cd55 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:40:59.316003 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-f09b1d8d-fba7-4c2b-9d9a-a4dfc959cd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:40:59.439414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09b1d8d-fba7-4c2b-9d9a-a4dfc959cd55 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:40:59.522033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41558c7e-e7d1-4c42-a288-3ac3dddc21e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:40:59.523495 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41558c7e-e7d1-4c42-a288-3ac3dddc21e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:40:59.523938 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41558c7e-e7d1-4c42-a288-3ac3dddc21e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:40:59.524411 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41558c7e-e7d1-4c42-a288-3ac3dddc21e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:40:59.526278 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-41558c7e-e7d1-4c42-a288-3ac3dddc21e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:40:59.584519 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:40:59.643193 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41558c7e-e7d1-4c42-a288-3ac3dddc21e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:40:59.644640 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41558c7e-e7d1-4c42-a288-3ac3dddc21e0 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:40:59.664622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 652/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:40:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:41:02.180434 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92cc5f-45cb-4566-8df8-d1008f983b05 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:02.181017 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92cc5f-45cb-4566-8df8-d1008f983b05 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:02.181251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92cc5f-45cb-4566-8df8-d1008f983b05 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:02.182820 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 660/1312] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:41:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:41:02.188358 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ce5af-5b45-4754-aae3-886aee333723 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:02.188872 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ce5af-5b45-4754-aae3-886aee333723 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:02.189295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ce5af-5b45-4754-aae3-886aee333723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:02.282443 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-315ce5af-5b45-4754-aae3-886aee333723 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:41:02.283802 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-315ce5af-5b45-4754-aae3-886aee333723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:02.407284 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 653/1313] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:41:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:41:02.438128 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef92cc5f-45cb-4566-8df8-d1008f983b05 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:41:02.439630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ef92cc5f-45cb-4566-8df8-d1008f983b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:02.564715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef92cc5f-45cb-4566-8df8-d1008f983b05 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:41:02.642673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:41:02.645648 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e3b26-33d8-4507-ad9e-624ad6558c58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:41:02.646341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e3b26-33d8-4507-ad9e-624ad6558c58 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:02.646624 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e3b26-33d8-4507-ad9e-624ad6558c58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:02.646899 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e3b26-33d8-4507-ad9e-624ad6558c58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:02.648790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ef0e3b26-33d8-4507-ad9e-624ad6558c58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:02.770331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e3b26-33d8-4507-ad9e-624ad6558c58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:41:02.772775 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e3b26-33d8-4507-ad9e-624ad6558c58 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:41:02.793195 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 661/1314] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:41:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:41:05.670480 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456efdd5-8bc7-4099-bf9a-c140e7ac5de7 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:05.671011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456efdd5-8bc7-4099-bf9a-c140e7ac5de7 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:05.671011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456efdd5-8bc7-4099-bf9a-c140e7ac5de7 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:05.672414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 654/1315] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:41:05.677535 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39063ca6-c27b-418a-a1e1-6686e587e359 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:05.677933 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39063ca6-c27b-418a-a1e1-6686e587e359 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:05.678106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39063ca6-c27b-418a-a1e1-6686e587e359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:05.742235 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-39063ca6-c27b-418a-a1e1-6686e587e359 None None] MFA Rules not processed for user `02e527c314c64f03bbc6d51e8e772a09`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:41:05.743451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-39063ca6-c27b-418a-a1e1-6686e587e359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:05.794214 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:41:05.838056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 662/1316] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:41:05 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:41:05.878172 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456efdd5-8bc7-4099-bf9a-c140e7ac5de7 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:41:05.879693 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-456efdd5-8bc7-4099-bf9a-c140e7ac5de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:05.889388 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:41:05.985833 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456efdd5-8bc7-4099-bf9a-c140e7ac5de7 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:41:06.059188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f423d607-fb0b-4717-8e3b-05741ca9173a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:41:06.059770 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423d607-fb0b-4717-8e3b-05741ca9173a service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:06.060052 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423d607-fb0b-4717-8e3b-05741ca9173a service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:06.060327 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f423d607-fb0b-4717-8e3b-05741ca9173a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:06.061977 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-f423d607-fb0b-4717-8e3b-05741ca9173a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:06.123489 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:41:06.170675 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f423d607-fb0b-4717-8e3b-05741ca9173a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:41:06.172117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f423d607-fb0b-4717-8e3b-05741ca9173a service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:41:06.189936 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 655/1317] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:41:05 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:41:09.291650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1eefa-68de-4335-a025-74823571b522 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:09.292180 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1eefa-68de-4335-a025-74823571b522 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:09.292180 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1eefa-68de-4335-a025-74823571b522 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:09.293513 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 663/1318] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:41:09.298792 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535f0f6-d217-40b6-8e22-d8e4f3c4b852 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:09.299174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535f0f6-d217-40b6-8e22-d8e4f3c4b852 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:09.299360 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535f0f6-d217-40b6-8e22-d8e4f3c4b852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:09.368418 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-3535f0f6-d217-40b6-8e22-d8e4f3c4b852 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:41:09.369667 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-3535f0f6-d217-40b6-8e22-d8e4f3c4b852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:09.476314 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 656/1319] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:41:09 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:41:09.514393 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1eefa-68de-4335-a025-74823571b522 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:41:09.515903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-7bd1eefa-68de-4335-a025-74823571b522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:09.617234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1eefa-68de-4335-a025-74823571b522 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:41:09.689637 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f49820c-6043-4834-bb67-f683a0960a76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:41:09.690287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49820c-6043-4834-bb67-f683a0960a76 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:09.690570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49820c-6043-4834-bb67-f683a0960a76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:09.690847 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f49820c-6043-4834-bb67-f683a0960a76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:09.692469 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-8f49820c-6043-4834-bb67-f683a0960a76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:09.700379 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:41:09.800283 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49820c-6043-4834-bb67-f683a0960a76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:41:09.801760 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f49820c-6043-4834-bb67-f683a0960a76 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:41:09.821196 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 664/1320] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:41:09 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:41:13.091778 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec85b73-41e6-4f38-ad59-1d953d5575a4 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:13.092317 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec85b73-41e6-4f38-ad59-1d953d5575a4 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:13.092317 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec85b73-41e6-4f38-ad59-1d953d5575a4 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:13.093891 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 657/1321] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:41:13.099515 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef968f9c-d64b-4e54-9944-866332db820e None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:13.100586 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef968f9c-d64b-4e54-9944-866332db820e None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:13.100758 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef968f9c-d64b-4e54-9944-866332db820e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:13.165108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-ef968f9c-d64b-4e54-9944-866332db820e None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:41:13.165725 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ef968f9c-d64b-4e54-9944-866332db820e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:13.226009 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:41:13.265789 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 665/1322] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:41:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:41:13.316025 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ebc12-42f7-4fb3-a516-e00a0d2e8f6f None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:13.316403 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ebc12-42f7-4fb3-a516-e00a0d2e8f6f None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:13.316753 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ebc12-42f7-4fb3-a516-e00a0d2e8f6f None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:13.318228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 658/1323] 10.222.0.13 () {54 vars in 964 bytes} [Fri May 27 19:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:41:13.325430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76b2d3-85cc-4df6-aa77-7bfbfdeb9e49 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:13.325746 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76b2d3-85cc-4df6-aa77-7bfbfdeb9e49 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:13.326028 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76b2d3-85cc-4df6-aa77-7bfbfdeb9e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:13.406516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-8b76b2d3-85cc-4df6-aa77-7bfbfdeb9e49 None None] MFA Rules not processed for user `dd34f15400c4476bbb2161dbe1b8b2b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:41:13.407785 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-8b76b2d3-85cc-4df6-aa77-7bfbfdeb9e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:13.491491 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 666/1324] 10.222.0.13 () {58 vars in 1061 bytes} [Fri May 27 19:41:13 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:41:13.500327 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ebc12-42f7-4fb3-a516-e00a0d2e8f6f None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:41:13.501880 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-c77ebc12-42f7-4fb3-a516-e00a0d2e8f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:13.526195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:41:13.608972 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ebc12-42f7-4fb3-a516-e00a0d2e8f6f None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:41:13.683250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338c5673-0c23-4862-a866-8bab7598366b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:41:13.683902 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c5673-0c23-4862-a866-8bab7598366b service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:13.684182 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c5673-0c23-4862-a866-8bab7598366b service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:13.684454 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338c5673-0c23-4862-a866-8bab7598366b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:13.687206 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-338c5673-0c23-4862-a866-8bab7598366b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:13.760731 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:41:13.797451 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c5673-0c23-4862-a866-8bab7598366b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:41:13.798827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338c5673-0c23-4862-a866-8bab7598366b service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:41:13.817083 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 659/1325] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:41:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:41:16.332526 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44bf42-4030-4500-a34a-5907b89a17c0 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:16.333150 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44bf42-4030-4500-a34a-5907b89a17c0 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:16.333150 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf44bf42-4030-4500-a34a-5907b89a17c0 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:16.335161 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 667/1326] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:41:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:41:16.340646 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7a289-097c-402c-a9d3-a29c85893adb None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:16.341117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7a289-097c-402c-a9d3-a29c85893adb None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:16.341257 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da7a289-097c-402c-a9d3-a29c85893adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:16.406128 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-9da7a289-097c-402c-a9d3-a29c85893adb None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:41:16.407309 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-9da7a289-097c-402c-a9d3-a29c85893adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:16.503285 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 660/1327] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:41:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:41:16.539601 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44bf42-4030-4500-a34a-5907b89a17c0 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:41:16.541227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-cf44bf42-4030-4500-a34a-5907b89a17c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:16.602349 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:41:16.661127 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf44bf42-4030-4500-a34a-5907b89a17c0 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:41:16.740508 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2898b9db-d3e5-4998-b439-5e4706549cdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:41:16.741253 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898b9db-d3e5-4998-b439-5e4706549cdc service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:16.741534 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898b9db-d3e5-4998-b439-5e4706549cdc service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:16.741810 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898b9db-d3e5-4998-b439-5e4706549cdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:16.743436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-2898b9db-d3e5-4998-b439-5e4706549cdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:16.852753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:41:16.855054 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2898b9db-d3e5-4998-b439-5e4706549cdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:41:16.856513 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2898b9db-d3e5-4998-b439-5e4706549cdc service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:41:16.874508 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 668/1328] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:41:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:41:19.167923 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779d8d8-0f58-4b88-843e-bfa8fd89622a None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:19.168458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779d8d8-0f58-4b88-843e-bfa8fd89622a None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:19.168458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e779d8d8-0f58-4b88-843e-bfa8fd89622a None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:19.169903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 661/1329] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:41:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:41:19.175220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619ad15-209a-4a1d-936d-51d6141dbcf9 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:19.175637 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619ad15-209a-4a1d-936d-51d6141dbcf9 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:19.175783 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619ad15-209a-4a1d-936d-51d6141dbcf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:19.240746 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-8619ad15-209a-4a1d-936d-51d6141dbcf9 None None] MFA Rules not processed for user `02e527c314c64f03bbc6d51e8e772a09`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:41:19.241888 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-8619ad15-209a-4a1d-936d-51d6141dbcf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:19.339696 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 669/1330] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:41:19 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:41:19.377054 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779d8d8-0f58-4b88-843e-bfa8fd89622a None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:41:19.378574 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-e779d8d8-0f58-4b88-843e-bfa8fd89622a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:19.409624 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:41:19.481709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e779d8d8-0f58-4b88-843e-bfa8fd89622a None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:41:19.558254 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498cac1c-3e04-4419-8023-4ff583c64358 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:41:19.558908 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cac1c-3e04-4419-8023-4ff583c64358 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:19.559193 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cac1c-3e04-4419-8023-4ff583c64358 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:19.559470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498cac1c-3e04-4419-8023-4ff583c64358 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:19.561128 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-498cac1c-3e04-4419-8023-4ff583c64358 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:19.640657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:41:19.673189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498cac1c-3e04-4419-8023-4ff583c64358 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:41:19.674644 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498cac1c-3e04-4419-8023-4ff583c64358 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:41:19.693903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 662/1331] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:41:19 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:41:22.321792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ab526-a683-43eb-a6bd-1b6588e3371a None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:22.322339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ab526-a683-43eb-a6bd-1b6588e3371a None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:22.322339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ab526-a683-43eb-a6bd-1b6588e3371a None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:22.323713 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 670/1332] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:41:22.329119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056623e-1118-4ff3-94c7-5906fd410a45 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:22.329538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056623e-1118-4ff3-94c7-5906fd410a45 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:22.329685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056623e-1118-4ff3-94c7-5906fd410a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:22.399729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-4056623e-1118-4ff3-94c7-5906fd410a45 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:41:22.400929 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-4056623e-1118-4ff3-94c7-5906fd410a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:22.498008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 663/1333] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:41:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:41:22.534432 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ab526-a683-43eb-a6bd-1b6588e3371a None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:41:22.535939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-7e7ab526-a683-43eb-a6bd-1b6588e3371a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:22.604366 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:41:22.638607 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ab526-a683-43eb-a6bd-1b6588e3371a None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:41:22.711125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad449ac-4f59-482b-8e82-599cae8f92cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:41:22.711578 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad449ac-4f59-482b-8e82-599cae8f92cf service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:22.711859 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad449ac-4f59-482b-8e82-599cae8f92cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:22.712133 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad449ac-4f59-482b-8e82-599cae8f92cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:22.713784 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-9ad449ac-4f59-482b-8e82-599cae8f92cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:22.822009 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad449ac-4f59-482b-8e82-599cae8f92cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:41:22.823376 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad449ac-4f59-482b-8e82-599cae8f92cf service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:41:22.833124 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:41:22.841879 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 671/1334] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:41:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:41:25.116914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c96200-a8c8-4e1c-ba63-febcd821b508 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:25.117482 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c96200-a8c8-4e1c-ba63-febcd821b508 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:25.117580 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c96200-a8c8-4e1c-ba63-febcd821b508 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:25.118908 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 664/1335] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:41:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:41:25.124385 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccf1e9-fae4-416b-b024-0a4fe677c2c0 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:25.124855 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccf1e9-fae4-416b-b024-0a4fe677c2c0 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:25.124993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccf1e9-fae4-416b-b024-0a4fe677c2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:25.189188 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-5cccf1e9-fae4-416b-b024-0a4fe677c2c0 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:41:25.190403 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-5cccf1e9-fae4-416b-b024-0a4fe677c2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:25.285031 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 672/1336] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:41:25.326114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c96200-a8c8-4e1c-ba63-febcd821b508 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:41:25.327649 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-c3c96200-a8c8-4e1c-ba63-febcd821b508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:25.367904 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:41:25.431577 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c96200-a8c8-4e1c-ba63-febcd821b508 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:41:25.503804 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bde303-6535-47e5-9d18-21c7f6d21602 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:41:25.504454 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bde303-6535-47e5-9d18-21c7f6d21602 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:25.504741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bde303-6535-47e5-9d18-21c7f6d21602 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:25.505105 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bde303-6535-47e5-9d18-21c7f6d21602 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:25.506759 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-a9bde303-6535-47e5-9d18-21c7f6d21602 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:25.595355 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:41:25.623275 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bde303-6535-47e5-9d18-21c7f6d21602 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:41:25.625060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bde303-6535-47e5-9d18-21c7f6d21602 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:41:25.642462 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 665/1337] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:41:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:41:28.482763 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debd087-471d-420e-88e9-614df3d753dd None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:28.483298 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debd087-471d-420e-88e9-614df3d753dd None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:28.483392 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debd087-471d-420e-88e9-614df3d753dd None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:28.484740 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 673/1338] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:41:28.490461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e9f1d-4143-4453-81f2-ccea19b2c018 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:28.490906 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e9f1d-4143-4453-81f2-ccea19b2c018 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:28.491010 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e9f1d-4143-4453-81f2-ccea19b2c018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:28.570950 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-8f8e9f1d-4143-4453-81f2-ccea19b2c018 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:41:28.572148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-8f8e9f1d-4143-4453-81f2-ccea19b2c018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:28.666291 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 666/1339] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:41:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:41:28.701642 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debd087-471d-420e-88e9-614df3d753dd None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:41:28.703197 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-6debd087-471d-420e-88e9-614df3d753dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:28.779826 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:41:28.808312 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debd087-471d-420e-88e9-614df3d753dd None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:41:28.885255 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d6cd24-25bd-4097-a340-6886b71907ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:41:28.885906 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6cd24-25bd-4097-a340-6886b71907ed service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:28.886193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6cd24-25bd-4097-a340-6886b71907ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:28.886474 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d6cd24-25bd-4097-a340-6886b71907ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:28.888540 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-84d6cd24-25bd-4097-a340-6886b71907ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:28.999894 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d6cd24-25bd-4097-a340-6886b71907ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:41:29.001392 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d6cd24-25bd-4097-a340-6886b71907ed service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:41:29.023068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 674/1340] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:41:28 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:41:33.398629 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f726d-66c3-420b-927d-02879b883274 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:33.399189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f726d-66c3-420b-927d-02879b883274 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:33.399189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f726d-66c3-420b-927d-02879b883274 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:33.400722 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 667/1341] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:41:33.405930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f8441-3667-4499-aeef-b47014cbfdc8 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:33.406372 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f8441-3667-4499-aeef-b47014cbfdc8 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:33.406562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f8441-3667-4499-aeef-b47014cbfdc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:33.412670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:41:33.454485 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-916f8441-3667-4499-aeef-b47014cbfdc8 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:41:33.455677 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-916f8441-3667-4499-aeef-b47014cbfdc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:33.535393 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 675/1342] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:41:33.573950 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f726d-66c3-420b-927d-02879b883274 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:41:33.576043 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-316f726d-66c3-420b-927d-02879b883274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:33.623491 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:41:33.654467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f726d-66c3-420b-927d-02879b883274 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:41:33.658709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf58365-bbaf-4337-ab12-ca8a193fcd1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:41:33.659357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf58365-bbaf-4337-ab12-ca8a193fcd1d service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:33.659640 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf58365-bbaf-4337-ab12-ca8a193fcd1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:33.659916 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf58365-bbaf-4337-ab12-ca8a193fcd1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:33.669292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf58365-bbaf-4337-ab12-ca8a193fcd1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:41:33.670717 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf58365-bbaf-4337-ab12-ca8a193fcd1d service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:41:33.680696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 668/1343] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:41:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:41:36.878691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fdf79-ba06-4e18-9a20-776cd1420481 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:36.879353 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fdf79-ba06-4e18-9a20-776cd1420481 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:36.879353 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fdf79-ba06-4e18-9a20-776cd1420481 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:36.880693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 676/1344] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:41:36.885928 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e71073-384f-48be-8ea6-b34cfc666c78 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:36.886397 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e71073-384f-48be-8ea6-b34cfc666c78 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:36.886529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e71073-384f-48be-8ea6-b34cfc666c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:36.915148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-33e71073-384f-48be-8ea6-b34cfc666c78 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:41:36.916344 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-33e71073-384f-48be-8ea6-b34cfc666c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:36.924143 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 669/1345] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:41:36 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:41:36.961528 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fdf79-ba06-4e18-9a20-776cd1420481 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:41:36.972276 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fdf79-ba06-4e18-9a20-776cd1420481 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:41:36.976402 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65488d-b8d0-4bc4-8798-97cea023a10f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:41:36.977069 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65488d-b8d0-4bc4-8798-97cea023a10f service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:36.977356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65488d-b8d0-4bc4-8798-97cea023a10f service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:36.977635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65488d-b8d0-4bc4-8798-97cea023a10f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:36.986566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65488d-b8d0-4bc4-8798-97cea023a10f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:41:36.988022 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65488d-b8d0-4bc4-8798-97cea023a10f service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:41:36.998124 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 677/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:41:36 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:41:41.456237 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377547db-ba98-49be-bcb1-5663184ca6c7 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:41.456784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377547db-ba98-49be-bcb1-5663184ca6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:41.456878 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377547db-ba98-49be-bcb1-5663184ca6c7 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:41.458935 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 670/1347] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:41:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:41:41.464286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4057e7-f901-4a9e-8233-af4f66f39c9f None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:41.464714 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4057e7-f901-4a9e-8233-af4f66f39c9f None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:41.465048 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4057e7-f901-4a9e-8233-af4f66f39c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:41.483705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-7f4057e7-f901-4a9e-8233-af4f66f39c9f None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:41:41.484924 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-7f4057e7-f901-4a9e-8233-af4f66f39c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:41.492760 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 678/1348] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:41:41.530691 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377547db-ba98-49be-bcb1-5663184ca6c7 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:41:41.542425 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377547db-ba98-49be-bcb1-5663184ca6c7 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:41:41.545943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bae72e-1eba-402a-b315-b24612d880d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:41:41.546574 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bae72e-1eba-402a-b315-b24612d880d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:41.546855 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bae72e-1eba-402a-b315-b24612d880d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:41.547130 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bae72e-1eba-402a-b315-b24612d880d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:41.556543 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bae72e-1eba-402a-b315-b24612d880d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:41:41.558048 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bae72e-1eba-402a-b315-b24612d880d6 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:41:41.567430 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 671/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:41:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:41:46.963260 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3590e7-3231-4a45-a92b-9a6b43c6f841 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:46.964342 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3590e7-3231-4a45-a92b-9a6b43c6f841 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:46.964848 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3590e7-3231-4a45-a92b-9a6b43c6f841 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:46.967111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 679/1350] 10.222.0.13 () {54 vars in 921 bytes} [Fri May 27 19:41:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:41:46.973335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff08fc-22d3-4fdc-8136-fe2c393f4fa8 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:46.975119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff08fc-22d3-4fdc-8136-fe2c393f4fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:46.975641 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff08fc-22d3-4fdc-8136-fe2c393f4fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:47.100037 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-6bff08fc-22d3-4fdc-8136-fe2c393f4fa8 None None] MFA Rules not processed for user `ffc2ed13a0c84f128b795002892efc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:41:47.101924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-6bff08fc-22d3-4fdc-8136-fe2c393f4fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:47.288149 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 672/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Fri May 27 19:41:46 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:41:51.479153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736880f-7b98-4a4f-bf68-a3a83afa6928 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:51.480557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=52 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:41:51.481238 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736880f-7b98-4a4f-bf68-a3a83afa6928 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:51.481812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2736880f-7b98-4a4f-bf68-a3a83afa6928 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:51.483524 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 680/1352] 10.222.0.13 () {54 vars in 921 bytes} [Fri May 27 19:41:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:41:51.489558 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b3818-49c0-43a2-be27-0e9d05793ad8 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:51.490054 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b3818-49c0-43a2-be27-0e9d05793ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:51.490138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47b3818-49c0-43a2-be27-0e9d05793ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:51.509510 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-f47b3818-49c0-43a2-be27-0e9d05793ad8 None None] MFA Rules not processed for user `ffc2ed13a0c84f128b795002892efc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:41:51.511695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-f47b3818-49c0-43a2-be27-0e9d05793ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:51.524546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 673/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Fri May 27 19:41:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:41:53.138340 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1fb3e-1f84-4593-a386-6d2f9d547ea6 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:53.138858 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1fb3e-1f84-4593-a386-6d2f9d547ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:53.138858 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1fb3e-1f84-4593-a386-6d2f9d547ea6 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:53.140351 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 681/1354] 10.222.0.13 () {54 vars in 921 bytes} [Fri May 27 19:41:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:41:53.145467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d27a7-a9ad-4b33-b84e-4dffc67c0bf5 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:53.145901 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d27a7-a9ad-4b33-b84e-4dffc67c0bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:53.146029 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d27a7-a9ad-4b33-b84e-4dffc67c0bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:53.164708 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-d07d27a7-a9ad-4b33-b84e-4dffc67c0bf5 None None] MFA Rules not processed for user `ffc2ed13a0c84f128b795002892efc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:41:53.165933 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d07d27a7-a9ad-4b33-b84e-4dffc67c0bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:53.174412 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 674/1355] 10.222.0.13 () {58 vars in 1018 bytes} [Fri May 27 19:41:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:41:57.021542 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a46a7-4ddc-4c92-a09e-4abcca8fbc9c None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:57.022013 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a46a7-4ddc-4c92-a09e-4abcca8fbc9c None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:57.022097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a46a7-4ddc-4c92-a09e-4abcca8fbc9c None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:57.024683 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 682/1356] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:41:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:41:57.028662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31210c1-4f38-433a-84ab-12b69a6efb9b None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:57.029119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31210c1-4f38-433a-84ab-12b69a6efb9b None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:57.029303 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31210c1-4f38-433a-84ab-12b69a6efb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:57.047383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-c31210c1-4f38-433a-84ab-12b69a6efb9b None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:41:57.048557 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-c31210c1-4f38-433a-84ab-12b69a6efb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:57.057021 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 675/1357] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:41:57.110252 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485286c-3875-4385-a40b-b5383a16c21c None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:57.110806 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485286c-3875-4385-a40b-b5383a16c21c None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:57.111301 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485286c-3875-4385-a40b-b5383a16c21c None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:57.113672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 683/1358] 10.222.0.13 () {54 vars in 961 bytes} [Fri May 27 19:41:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:41:57.123115 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b0ed7-6f58-4bc9-8fff-2bfa422123d3 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:57.124197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b0ed7-6f58-4bc9-8fff-2bfa422123d3 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:57.125445 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b0ed7-6f58-4bc9-8fff-2bfa422123d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:57.165209 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:41:57.179708 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-5c8b0ed7-6f58-4bc9-8fff-2bfa422123d3 None None] MFA Rules not processed for user `7542e1fd160e4b2c9d25accf19aad19f`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:41:57.181882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-5c8b0ed7-6f58-4bc9-8fff-2bfa422123d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:57.275839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 676/1359] 10.222.0.13 () {58 vars in 1058 bytes} [Fri May 27 19:41:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:41:57.276794 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485286c-3875-4385-a40b-b5383a16c21c None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:41:57.291547 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485286c-3875-4385-a40b-b5383a16c21c None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:41:57.297277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35c4122-4a92-41e6-b925-d1912d4ec55a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:41:57.297911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c4122-4a92-41e6-b925-d1912d4ec55a service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:57.298194 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c4122-4a92-41e6-b925-d1912d4ec55a service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:57.298486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35c4122-4a92-41e6-b925-d1912d4ec55a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:57.308491 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c4122-4a92-41e6-b925-d1912d4ec55a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:41:57.310679 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35c4122-4a92-41e6-b925-d1912d4ec55a service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:41:57.325304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 684/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:41:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:41:59.779847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eae1d7-98e6-46f2-9b09-fc5545579393 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:59.780409 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eae1d7-98e6-46f2-9b09-fc5545579393 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:59.780409 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eae1d7-98e6-46f2-9b09-fc5545579393 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:59.781984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 677/1361] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:41:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:41:59.787675 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e7d8f-7779-4e00-b265-f1a97ca0bcac None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:59.788127 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e7d8f-7779-4e00-b265-f1a97ca0bcac None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:59.788211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e7d8f-7779-4e00-b265-f1a97ca0bcac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:59.806938 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-ed3e7d8f-7779-4e00-b265-f1a97ca0bcac None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:41:59.808055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ed3e7d8f-7779-4e00-b265-f1a97ca0bcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:41:59.816356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 685/1362] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:41:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:41:59.860866 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eae1d7-98e6-46f2-9b09-fc5545579393 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:41:59.877857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eae1d7-98e6-46f2-9b09-fc5545579393 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:41:59.884429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bfabbc-bfc4-45f2-a90a-e3a066a207d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:41:59.885113 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfabbc-bfc4-45f2-a90a-e3a066a207d4 service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:41:59.885401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfabbc-bfc4-45f2-a90a-e3a066a207d4 service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:41:59.885675 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bfabbc-bfc4-45f2-a90a-e3a066a207d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:41:59.895305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfabbc-bfc4-45f2-a90a-e3a066a207d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:41:59.896904 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bfabbc-bfc4-45f2-a90a-e3a066a207d4 service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:41:59.907976 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 678/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:41:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:42:02.588319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba28b73-8b8c-4a56-a491-29cd783981eb None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:02.589348 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba28b73-8b8c-4a56-a491-29cd783981eb None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:02.589631 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba28b73-8b8c-4a56-a491-29cd783981eb None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:02.591423 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 686/1364] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:42:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:02.597452 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff345a7-2b2e-49b9-82b4-1162f983a3bb None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:02.597879 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff345a7-2b2e-49b9-82b4-1162f983a3bb None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:02.598013 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff345a7-2b2e-49b9-82b4-1162f983a3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:02.616404 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-6ff345a7-2b2e-49b9-82b4-1162f983a3bb None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:02.617652 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-6ff345a7-2b2e-49b9-82b4-1162f983a3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:02.641809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 679/1365] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:42:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:02.689236 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a7e68-ff1f-4090-8d30-54c67d07453c None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:02.690069 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a7e68-ff1f-4090-8d30-54c67d07453c None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:02.690350 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a7e68-ff1f-4090-8d30-54c67d07453c None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:02.691729 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 687/1366] 10.222.0.13 () {54 vars in 961 bytes} [Fri May 27 19:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:02.697110 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8b3b4-66f0-4760-b9cd-89a42e6b437a None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:02.697410 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8b3b4-66f0-4760-b9cd-89a42e6b437a None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:02.697693 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8b3b4-66f0-4760-b9cd-89a42e6b437a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:02.716325 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-7ac8b3b4-66f0-4760-b9cd-89a42e6b437a None None] MFA Rules not processed for user `7542e1fd160e4b2c9d25accf19aad19f`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:02.717660 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-7ac8b3b4-66f0-4760-b9cd-89a42e6b437a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:02.727514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 680/1367] 10.222.0.13 () {58 vars in 1058 bytes} [Fri May 27 19:42:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:02.735301 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a7e68-ff1f-4090-8d30-54c67d07453c None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:42:02.749718 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a7e68-ff1f-4090-8d30-54c67d07453c None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:42:02.755734 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eba832c-bf02-4047-a5fc-1d3695e626b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:42:02.756373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba832c-bf02-4047-a5fc-1d3695e626b7 service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:02.756660 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba832c-bf02-4047-a5fc-1d3695e626b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:02.756965 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eba832c-bf02-4047-a5fc-1d3695e626b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:02.765084 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba832c-bf02-4047-a5fc-1d3695e626b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:42:02.766682 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eba832c-bf02-4047-a5fc-1d3695e626b7 service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:42:02.776856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 688/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:42:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:42:05.198457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85518f2b-fd53-4ff4-8924-24936b21a54d None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:05.198986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85518f2b-fd53-4ff4-8924-24936b21a54d None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:05.198986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85518f2b-fd53-4ff4-8924-24936b21a54d None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:05.200372 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 681/1369] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:05.205786 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8e569-5b42-43ad-b975-6346c17f6954 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:05.206212 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8e569-5b42-43ad-b975-6346c17f6954 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:05.206343 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8e569-5b42-43ad-b975-6346c17f6954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:05.225856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-98c8e569-5b42-43ad-b975-6346c17f6954 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:05.227290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-98c8e569-5b42-43ad-b975-6346c17f6954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:05.235273 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 689/1370] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:05.281113 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85518f2b-fd53-4ff4-8924-24936b21a54d None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:42:05.296297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85518f2b-fd53-4ff4-8924-24936b21a54d None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:42:05.302957 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d27b64-5d7d-4136-bcd6-bed4d1cdd325 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:42:05.305522 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d27b64-5d7d-4136-bcd6-bed4d1cdd325 service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:05.305522 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d27b64-5d7d-4136-bcd6-bed4d1cdd325 service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:05.305522 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d27b64-5d7d-4136-bcd6-bed4d1cdd325 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:05.313279 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d27b64-5d7d-4136-bcd6-bed4d1cdd325 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:42:05.314968 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d27b64-5d7d-4136-bcd6-bed4d1cdd325 service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:42:05.326312 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 682/1371] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:42:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:42:05.374004 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24826c-6aa6-43e4-8897-793753d4f131 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:05.374390 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24826c-6aa6-43e4-8897-793753d4f131 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:05.374680 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a24826c-6aa6-43e4-8897-793753d4f131 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:05.376090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 690/1372] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:05.382357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5c3f1-d797-4d7a-8dbc-39b2945702dd None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:05.382647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5c3f1-d797-4d7a-8dbc-39b2945702dd None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:05.382929 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5c3f1-d797-4d7a-8dbc-39b2945702dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:05.402150 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-0ff5c3f1-d797-4d7a-8dbc-39b2945702dd None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:05.403371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-0ff5c3f1-d797-4d7a-8dbc-39b2945702dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:05.412135 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 683/1373] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:42:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:05.743691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490f825-ed9d-48f7-a900-64c061281278 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:05.743982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490f825-ed9d-48f7-a900-64c061281278 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:05.744266 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490f825-ed9d-48f7-a900-64c061281278 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:05.745915 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 691/1374] 10.222.0.13 () {54 vars in 963 bytes} [Fri May 27 19:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:05.752007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d981d-fb6c-423f-8cab-b6b6f301c016 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:05.752306 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d981d-fb6c-423f-8cab-b6b6f301c016 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:05.752589 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d981d-fb6c-423f-8cab-b6b6f301c016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:05.803984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-d52d981d-fb6c-423f-8cab-b6b6f301c016 None None] MFA Rules not processed for user `7b7e90ca75d643cbb0e6d6791fc30b45`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:05.805277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d52d981d-fb6c-423f-8cab-b6b6f301c016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:05.869339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 684/1375] 10.222.0.13 () {58 vars in 1060 bytes} [Fri May 27 19:42:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:05.877659 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1490f825-ed9d-48f7-a900-64c061281278 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:42:05.888638 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1490f825-ed9d-48f7-a900-64c061281278 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:42:05.892808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c24cd0c-8d38-4916-9d1a-9dd8623e9274 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:42:05.893600 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24cd0c-8d38-4916-9d1a-9dd8623e9274 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:05.893889 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24cd0c-8d38-4916-9d1a-9dd8623e9274 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:05.894167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c24cd0c-8d38-4916-9d1a-9dd8623e9274 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:05.906230 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c24cd0c-8d38-4916-9d1a-9dd8623e9274 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:42:05.907750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c24cd0c-8d38-4916-9d1a-9dd8623e9274 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:42:05.919119 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 692/1376] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:42:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:42:07.769542 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d52a03-4a04-4739-9751-6253a0b99ae5 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:07.770095 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d52a03-4a04-4739-9751-6253a0b99ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:07.770095 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d52a03-4a04-4739-9751-6253a0b99ae5 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:07.771552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 685/1377] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:07.777408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a74a59-8eff-4e1f-bbd6-274c71161c49 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:07.777926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a74a59-8eff-4e1f-bbd6-274c71161c49 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:07.778044 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a74a59-8eff-4e1f-bbd6-274c71161c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:07.796849 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-65a74a59-8eff-4e1f-bbd6-274c71161c49 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:07.797976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-65a74a59-8eff-4e1f-bbd6-274c71161c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:07.806106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 693/1378] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:07.851674 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d52a03-4a04-4739-9751-6253a0b99ae5 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:42:07.868302 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d52a03-4a04-4739-9751-6253a0b99ae5 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:42:07.873934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee70fd-be38-4343-9e41-dbe5f13bd401 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:42:07.874596 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee70fd-be38-4343-9e41-dbe5f13bd401 service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:07.875617 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee70fd-be38-4343-9e41-dbe5f13bd401 service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:07.875617 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee70fd-be38-4343-9e41-dbe5f13bd401 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:07.881265 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:42:07.885074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee70fd-be38-4343-9e41-dbe5f13bd401 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:42:07.886687 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee70fd-be38-4343-9e41-dbe5f13bd401 service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:42:07.896335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 686/1379] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:42:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:42:10.386255 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafddb4f-f9a2-472a-b7b3-36dfc6366a21 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:10.387142 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafddb4f-f9a2-472a-b7b3-36dfc6366a21 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:10.387613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafddb4f-f9a2-472a-b7b3-36dfc6366a21 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:10.389046 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 694/1380] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:42:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:10.395072 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7225ea-e89a-407d-8a53-0cfbd2569205 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:10.396035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7225ea-e89a-407d-8a53-0cfbd2569205 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:10.396542 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7225ea-e89a-407d-8a53-0cfbd2569205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:10.418058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-8f7225ea-e89a-407d-8a53-0cfbd2569205 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:10.419983 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-8f7225ea-e89a-407d-8a53-0cfbd2569205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:10.442626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 687/1381] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:42:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:10.474518 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafddb4f-f9a2-472a-b7b3-36dfc6366a21 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:42:10.489729 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafddb4f-f9a2-472a-b7b3-36dfc6366a21 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:42:10.495286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224835c-187e-4d11-bcf3-32141216ddf0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:42:10.495916 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224835c-187e-4d11-bcf3-32141216ddf0 service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:10.496202 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224835c-187e-4d11-bcf3-32141216ddf0 service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:10.496478 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224835c-187e-4d11-bcf3-32141216ddf0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:10.505532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a224835c-187e-4d11-bcf3-32141216ddf0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:42:10.507119 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a224835c-187e-4d11-bcf3-32141216ddf0 service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:42:10.517497 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 695/1382] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:42:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:42:12.900591 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b75f6-9809-4507-89e1-535e2aa7b0c3 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:12.901519 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b75f6-9809-4507-89e1-535e2aa7b0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:12.901808 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b75f6-9809-4507-89e1-535e2aa7b0c3 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:12.903688 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 688/1383] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:42:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 19:42:12.910036 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894e606-36fe-463b-afc6-09ab58129d75 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:12.910565 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894e606-36fe-463b-afc6-09ab58129d75 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:12.910805 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894e606-36fe-463b-afc6-09ab58129d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:12.930382 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-2894e606-36fe-463b-afc6-09ab58129d75 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:12.931700 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-2894e606-36fe-463b-afc6-09ab58129d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:12.940586 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 696/1384] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:42:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:12.984409 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b75f6-9809-4507-89e1-535e2aa7b0c3 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:42:12.999632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b75f6-9809-4507-89e1-535e2aa7b0c3 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:42:13.005341 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8569fa9-d5bc-4959-937b-dbf9aa0810a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:42:13.005997 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8569fa9-d5bc-4959-937b-dbf9aa0810a2 service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:13.006493 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8569fa9-d5bc-4959-937b-dbf9aa0810a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:13.006778 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8569fa9-d5bc-4959-937b-dbf9aa0810a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:13.015974 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8569fa9-d5bc-4959-937b-dbf9aa0810a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:42:13.018676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8569fa9-d5bc-4959-937b-dbf9aa0810a2 service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:42:13.028604 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 689/1385] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:42:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:42:15.424784 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbbe57-50db-4a0a-a15e-114b44fab55b None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:15.425478 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbbe57-50db-4a0a-a15e-114b44fab55b None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:15.425478 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cbbe57-50db-4a0a-a15e-114b44fab55b None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:15.426694 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 697/1386] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:42:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:15.432451 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02caa24a-1c1e-4e71-8e5f-c26568314753 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:15.433067 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02caa24a-1c1e-4e71-8e5f-c26568314753 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:15.433170 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02caa24a-1c1e-4e71-8e5f-c26568314753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:15.453506 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-02caa24a-1c1e-4e71-8e5f-c26568314753 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:15.454906 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-02caa24a-1c1e-4e71-8e5f-c26568314753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:15.463808 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 690/1387] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:42:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:15.510012 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbbe57-50db-4a0a-a15e-114b44fab55b None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:42:15.527721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cbbe57-50db-4a0a-a15e-114b44fab55b None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:42:15.534819 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e46c2-2370-41d9-b4e2-80f334ef919e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:42:15.535487 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e46c2-2370-41d9-b4e2-80f334ef919e service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:15.535781 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e46c2-2370-41d9-b4e2-80f334ef919e service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:15.536068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e46c2-2370-41d9-b4e2-80f334ef919e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:15.544807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9e46c2-2370-41d9-b4e2-80f334ef919e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:42:15.546459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9e46c2-2370-41d9-b4e2-80f334ef919e service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:42:15.556485 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 698/1388] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:42:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:42:17.951528 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a245677-00cf-42d6-9f1a-ea58aa784d34 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:17.952296 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a245677-00cf-42d6-9f1a-ea58aa784d34 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:17.952669 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a245677-00cf-42d6-9f1a-ea58aa784d34 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:17.955212 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 691/1389] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:42:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:17.961770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a2fa4-3910-47b6-9955-dca2c2bcb1d4 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:17.962201 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a2fa4-3910-47b6-9955-dca2c2bcb1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:17.962323 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a2fa4-3910-47b6-9955-dca2c2bcb1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:17.981334 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-e90a2fa4-3910-47b6-9955-dca2c2bcb1d4 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:17.982572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-e90a2fa4-3910-47b6-9955-dca2c2bcb1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:17.990994 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 699/1390] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:42:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:18.036562 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a245677-00cf-42d6-9f1a-ea58aa784d34 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:42:18.052477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a245677-00cf-42d6-9f1a-ea58aa784d34 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:42:18.058095 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338e0af-c400-4abc-8fbb-e89e280f7c07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:42:18.058760 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338e0af-c400-4abc-8fbb-e89e280f7c07 service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:18.059054 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338e0af-c400-4abc-8fbb-e89e280f7c07 service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:18.059343 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338e0af-c400-4abc-8fbb-e89e280f7c07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:18.068867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338e0af-c400-4abc-8fbb-e89e280f7c07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:42:18.070534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338e0af-c400-4abc-8fbb-e89e280f7c07 service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:42:18.080418 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 692/1391] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:42:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:42:20.444082 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2e6a0-138c-4233-854a-078aa40b04f1 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:20.444784 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2e6a0-138c-4233-854a-078aa40b04f1 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:20.444784 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2e6a0-138c-4233-854a-078aa40b04f1 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:20.446045 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 700/1392] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:20.451207 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac7086-4941-43ac-87c0-7331c9c3126d None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:20.451678 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac7086-4941-43ac-87c0-7331c9c3126d None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:20.451793 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac7086-4941-43ac-87c0-7331c9c3126d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:20.471074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-06ac7086-4941-43ac-87c0-7331c9c3126d None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:20.472259 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-06ac7086-4941-43ac-87c0-7331c9c3126d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:20.484140 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 693/1393] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:42:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:20.525025 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2e6a0-138c-4233-854a-078aa40b04f1 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:42:20.540363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2e6a0-138c-4233-854a-078aa40b04f1 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:42:20.545756 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d1ec90-2d65-4626-b0ad-6744dd078f98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:42:20.546397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1ec90-2d65-4626-b0ad-6744dd078f98 service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:20.546684 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1ec90-2d65-4626-b0ad-6744dd078f98 service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:20.546966 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d1ec90-2d65-4626-b0ad-6744dd078f98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:20.555138 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d1ec90-2d65-4626-b0ad-6744dd078f98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:42:20.556736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d1ec90-2d65-4626-b0ad-6744dd078f98 service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:42:20.566002 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 701/1394] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:42:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:42:22.063588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dd1ef-3d25-4ee8-9a86-e14e61ee5ca0 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:22.064287 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dd1ef-3d25-4ee8-9a86-e14e61ee5ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:22.064287 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183dd1ef-3d25-4ee8-9a86-e14e61ee5ca0 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:22.065560 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 694/1395] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:42:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:22.070872 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3d7e4-ff45-4170-8bad-7cc02efa2081 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:22.071311 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3d7e4-ff45-4170-8bad-7cc02efa2081 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:22.071511 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3d7e4-ff45-4170-8bad-7cc02efa2081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:22.077598 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=52 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:42:22.089337 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-d3f3d7e4-ff45-4170-8bad-7cc02efa2081 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:22.093787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d3f3d7e4-ff45-4170-8bad-7cc02efa2081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:22.098705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 702/1396] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:22.154907 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8bc0a-47ea-4694-90e5-79d3cdf738fa None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:22.155248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8bc0a-47ea-4694-90e5-79d3cdf738fa None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:22.155534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8bc0a-47ea-4694-90e5-79d3cdf738fa None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:22.157193 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 695/1397] 10.222.0.13 () {54 vars in 962 bytes} [Fri May 27 19:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:22.162112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093a892-e8bd-47d0-bd24-280f8c1945d3 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:22.162479 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093a892-e8bd-47d0-bd24-280f8c1945d3 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:22.162842 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093a892-e8bd-47d0-bd24-280f8c1945d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:22.211667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-4093a892-e8bd-47d0-bd24-280f8c1945d3 None None] MFA Rules not processed for user `1347c0bd4ada4139ad194d423958be6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:22.212847 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-4093a892-e8bd-47d0-bd24-280f8c1945d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:22.275258 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 703/1398] 10.222.0.13 () {58 vars in 1059 bytes} [Fri May 27 19:42:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:22.283817 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8bc0a-47ea-4694-90e5-79d3cdf738fa None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:42:22.294423 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8bc0a-47ea-4694-90e5-79d3cdf738fa None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:42:22.298831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bc1e84-2263-4b3b-9b9a-0b7a5cbd8d9b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1347c0bd4ada4139ad194d423958be6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:42:22.299593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc1e84-2263-4b3b-9b9a-0b7a5cbd8d9b service glance] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:22.299860 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc1e84-2263-4b3b-9b9a-0b7a5cbd8d9b service glance] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:22.300137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bc1e84-2263-4b3b-9b9a-0b7a5cbd8d9b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:22.309191 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc1e84-2263-4b3b-9b9a-0b7a5cbd8d9b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:42:22.310621 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bc1e84-2263-4b3b-9b9a-0b7a5cbd8d9b service glance] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:42:22.321623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 696/1399] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:42:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:42:22.879656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07b019-48ee-427f-ac18-0ac1c4703594 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:22.880698 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07b019-48ee-427f-ac18-0ac1c4703594 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:22.880698 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07b019-48ee-427f-ac18-0ac1c4703594 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:22.881960 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 704/1400] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:22.887071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e24b16-311d-49ef-a670-b39cde4bc10e None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:22.887371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e24b16-311d-49ef-a670-b39cde4bc10e None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:22.887661 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e24b16-311d-49ef-a670-b39cde4bc10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:22.907102 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-08e24b16-311d-49ef-a670-b39cde4bc10e None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:22.908212 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-08e24b16-311d-49ef-a670-b39cde4bc10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:22.916170 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 697/1401] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:22.961234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07b019-48ee-427f-ac18-0ac1c4703594 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:42:22.975997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07b019-48ee-427f-ac18-0ac1c4703594 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:42:22.981875 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b38e0b-14d6-4666-8166-9a8911971ff8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:42:22.982751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b38e0b-14d6-4666-8166-9a8911971ff8 service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:22.983163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b38e0b-14d6-4666-8166-9a8911971ff8 service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:22.983542 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b38e0b-14d6-4666-8166-9a8911971ff8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:22.993415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b38e0b-14d6-4666-8166-9a8911971ff8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:42:22.995390 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b38e0b-14d6-4666-8166-9a8911971ff8 service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:42:23.010598 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60feee6d-69e8-44c1-99b7-dd3d97e0ebca None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:23.010959 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60feee6d-69e8-44c1-99b7-dd3d97e0ebca None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:23.011286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 705/1402] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:42:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:42:23.011465 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60feee6d-69e8-44c1-99b7-dd3d97e0ebca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:23.061698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-60feee6d-69e8-44c1-99b7-dd3d97e0ebca None None] MFA Rules not processed for user `c700b0adb9f54947a88c1baa86accb9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:23.063385 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-60feee6d-69e8-44c1-99b7-dd3d97e0ebca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:23.128961 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 698/1403] 10.222.0.13 () {58 vars in 1009 bytes} [Fri May 27 19:42:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:42:23.174407 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d7523-b1ea-47b0-a42a-ca0bb7f954e8 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:23.174862 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d7523-b1ea-47b0-a42a-ca0bb7f954e8 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:23.174981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d7523-b1ea-47b0-a42a-ca0bb7f954e8 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:23.176334 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 706/1404] 10.222.0.13 () {54 vars in 942 bytes} [Fri May 27 19:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:23.182833 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03ebd8-9b5b-462d-a539-72fcd11a2956 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:23.183127 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03ebd8-9b5b-462d-a539-72fcd11a2956 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:23.183410 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03ebd8-9b5b-462d-a539-72fcd11a2956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:23.238434 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-df03ebd8-9b5b-462d-a539-72fcd11a2956 None None] MFA Rules not processed for user `37bc6d0abb454de19e1ab2f46bf4a593`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:23.241329 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-df03ebd8-9b5b-462d-a539-72fcd11a2956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:23.274479 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:42:23.321124 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 699/1405] 10.222.0.13 () {58 vars in 1039 bytes} [Fri May 27 19:42:23 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:23.332045 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d7523-b1ea-47b0-a42a-ca0bb7f954e8 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:42:23.346672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d7523-b1ea-47b0-a42a-ca0bb7f954e8 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:42:23.351738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c335f-af98-4f86-ae91-0dacf120d3e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37bc6d0abb454de19e1ab2f46bf4a593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:42:23.352410 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c335f-af98-4f86-ae91-0dacf120d3e7 service swift] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:23.352699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c335f-af98-4f86-ae91-0dacf120d3e7 service swift] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:23.353017 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c335f-af98-4f86-ae91-0dacf120d3e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:23.363324 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c335f-af98-4f86-ae91-0dacf120d3e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:42:23.365234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715c335f-af98-4f86-ae91-0dacf120d3e7 service swift] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:42:23.374346 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 707/1406] 10.222.0.13 () {60 vars in 1384 bytes} [Fri May 27 19:42:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 19:42:25.437999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7455c1-d823-4980-8629-365ea41466ab None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:25.438527 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7455c1-d823-4980-8629-365ea41466ab None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:25.438761 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7455c1-d823-4980-8629-365ea41466ab None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:25.440232 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 700/1407] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:42:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:25.445568 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0423cb-c842-44b5-a735-ddc614c1bec8 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:25.446415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0423cb-c842-44b5-a735-ddc614c1bec8 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:25.446705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0423cb-c842-44b5-a735-ddc614c1bec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:25.619514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-3f0423cb-c842-44b5-a735-ddc614c1bec8 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:25.620711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-3f0423cb-c842-44b5-a735-ddc614c1bec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:25.629650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 708/1408] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:42:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:25.683613 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7455c1-d823-4980-8629-365ea41466ab None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:42:25.698333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7455c1-d823-4980-8629-365ea41466ab None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:42:25.705919 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b97f0-e977-4bd9-bac8-422e2b918af1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:42:25.706589 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b97f0-e977-4bd9-bac8-422e2b918af1 service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:25.706871 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b97f0-e977-4bd9-bac8-422e2b918af1 service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:25.707148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b97f0-e977-4bd9-bac8-422e2b918af1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:25.716351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b97f0-e977-4bd9-bac8-422e2b918af1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:42:25.718237 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b97f0-e977-4bd9-bac8-422e2b918af1 service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:42:25.729195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 701/1409] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:42:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:42:35.613890 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=55 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:42:35.725191 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=23 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:42:48.155577 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7603e4-abe4-47ff-a428-1c1df5b64227 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:48.156242 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7603e4-abe4-47ff-a428-1c1df5b64227 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:48.156242 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7603e4-abe4-47ff-a428-1c1df5b64227 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:48.157556 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 709/1410] 10.222.0.13 () {54 vars in 918 bytes} [Fri May 27 19:42:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:48.162648 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0e70f-b958-4813-a2af-df0348afd118 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:48.163123 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0e70f-b958-4813-a2af-df0348afd118 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:48.163238 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0e70f-b958-4813-a2af-df0348afd118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:48.180745 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-aba0e70f-b958-4813-a2af-df0348afd118 None None] MFA Rules not processed for user `ffc2ed13a0c84f128b795002892efc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:48.181964 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-aba0e70f-b958-4813-a2af-df0348afd118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:48.190493 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 702/1411] 10.222.0.13 () {58 vars in 1015 bytes} [Fri May 27 19:42:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:48.205398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad19ff-7b20-4e44-84e9-8617d9e3f2ef None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:48.213080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad19ff-7b20-4e44-84e9-8617d9e3f2ef None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:48.213080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad19ff-7b20-4e44-84e9-8617d9e3f2ef None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:48.213080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 710/1412] 10.222.0.13 () {54 vars in 918 bytes} [Fri May 27 19:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:48.213559 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23385364-110d-43e7-93cc-af0f0d8d006d None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:48.213559 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23385364-110d-43e7-93cc-af0f0d8d006d None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:48.213559 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23385364-110d-43e7-93cc-af0f0d8d006d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:48.230922 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-23385364-110d-43e7-93cc-af0f0d8d006d None None] MFA Rules not processed for user `7542e1fd160e4b2c9d25accf19aad19f`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:48.232164 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-23385364-110d-43e7-93cc-af0f0d8d006d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:48.241132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 703/1413] 10.222.0.13 () {58 vars in 1015 bytes} [Fri May 27 19:42:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:50.914802 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3cd5a8-0264-4665-a194-b099a47c3c74 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:50.915365 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3cd5a8-0264-4665-a194-b099a47c3c74 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:50.915365 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3cd5a8-0264-4665-a194-b099a47c3c74 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:50.916615 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 19:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:50.922471 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b73a9c-d5ce-4ebf-9544-6f94e0708bbd None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:50.922919 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b73a9c-d5ce-4ebf-9544-6f94e0708bbd None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:50.923027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b73a9c-d5ce-4ebf-9544-6f94e0708bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:50.940562 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-95b73a9c-d5ce-4ebf-9544-6f94e0708bbd None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:50.941768 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-95b73a9c-d5ce-4ebf-9544-6f94e0708bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:50.949389 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 19:42:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:50.992198 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3cd5a8-0264-4665-a194-b099a47c3c74 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:42:51.006391 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3cd5a8-0264-4665-a194-b099a47c3c74 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:42:51.011669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244eb94-d504-49d7-9f75-a6b27e3cf976 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:42:51.012296 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244eb94-d504-49d7-9f75-a6b27e3cf976 service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:51.012576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244eb94-d504-49d7-9f75-a6b27e3cf976 service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:51.012851 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244eb94-d504-49d7-9f75-a6b27e3cf976 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:51.021026 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244eb94-d504-49d7-9f75-a6b27e3cf976 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:42:51.022595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244eb94-d504-49d7-9f75-a6b27e3cf976 service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:42:51.032505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 712/1416] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:42:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:42:52.017738 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343230b9-2d72-4576-ad76-cb30e012161c None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:52.018316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343230b9-2d72-4576-ad76-cb30e012161c None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:52.018316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343230b9-2d72-4576-ad76-cb30e012161c None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:52.019656 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 19:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:52.024648 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd1e42-ad6b-4a36-b9b1-f5a269016da9 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:52.025058 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd1e42-ad6b-4a36-b9b1-f5a269016da9 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:52.025237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fd1e42-ad6b-4a36-b9b1-f5a269016da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:52.043632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-45fd1e42-ad6b-4a36-b9b1-f5a269016da9 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:52.045072 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-45fd1e42-ad6b-4a36-b9b1-f5a269016da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:52.052826 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 19:42:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:52.219847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343230b9-2d72-4576-ad76-cb30e012161c None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:42:52.233593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343230b9-2d72-4576-ad76-cb30e012161c None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:42:52.239221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49806b6-6dc3-4500-bc6d-d765a73b4137 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:42:52.245071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49806b6-6dc3-4500-bc6d-d765a73b4137 service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:52.245071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49806b6-6dc3-4500-bc6d-d765a73b4137 service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:52.245071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49806b6-6dc3-4500-bc6d-d765a73b4137 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:52.249201 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49806b6-6dc3-4500-bc6d-d765a73b4137 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:42:52.250767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49806b6-6dc3-4500-bc6d-d765a73b4137 service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:42:52.259865 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 706/1419] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:42:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:42:53.195274 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7f677-31f3-4eac-8872-c92421b2ed50 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:53.195827 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7f677-31f3-4eac-8872-c92421b2ed50 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:53.195926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7f677-31f3-4eac-8872-c92421b2ed50 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:53.197326 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 19:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:53.203495 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2623f-40eb-4c40-a2e3-21870a886972 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:53.203950 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2623f-40eb-4c40-a2e3-21870a886972 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:53.204057 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce2623f-40eb-4c40-a2e3-21870a886972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:53.222396 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-9ce2623f-40eb-4c40-a2e3-21870a886972 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:53.227652 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-9ce2623f-40eb-4c40-a2e3-21870a886972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:53.231449 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 19:42:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:53.383632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7f677-31f3-4eac-8872-c92421b2ed50 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:42:53.397769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7f677-31f3-4eac-8872-c92421b2ed50 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:42:53.403158 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad46c81-b44e-4b15-b88b-cd1fe3f829ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:42:53.403925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad46c81-b44e-4b15-b88b-cd1fe3f829ad service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:53.404213 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad46c81-b44e-4b15-b88b-cd1fe3f829ad service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:53.404490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad46c81-b44e-4b15-b88b-cd1fe3f829ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:53.412890 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad46c81-b44e-4b15-b88b-cd1fe3f829ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:42:53.414490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad46c81-b44e-4b15-b88b-cd1fe3f829ad service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:42:53.424195 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 715/1422] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:42:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:42:54.332766 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e2851-635c-4a87-a893-9d98fd78800a None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:54.333369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e2851-635c-4a87-a893-9d98fd78800a None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:54.333369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0e2851-635c-4a87-a893-9d98fd78800a None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:54.334733 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 19:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:54.339645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7016c95-8ea7-4b1b-a213-3e5c7cd7f214 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:54.340056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7016c95-8ea7-4b1b-a213-3e5c7cd7f214 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:54.340194 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7016c95-8ea7-4b1b-a213-3e5c7cd7f214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:54.359118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-a7016c95-8ea7-4b1b-a213-3e5c7cd7f214 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:54.360285 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-a7016c95-8ea7-4b1b-a213-3e5c7cd7f214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:54.367831 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 19:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:54.517778 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e2851-635c-4a87-a893-9d98fd78800a None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:42:54.532152 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0e2851-635c-4a87-a893-9d98fd78800a None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:42:54.537430 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7c8042-b34a-4437-9117-ff4a74ddb84e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:42:54.538070 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c8042-b34a-4437-9117-ff4a74ddb84e service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:54.538355 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c8042-b34a-4437-9117-ff4a74ddb84e service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:54.538632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7c8042-b34a-4437-9117-ff4a74ddb84e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:54.546995 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c8042-b34a-4437-9117-ff4a74ddb84e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:42:54.548583 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7c8042-b34a-4437-9117-ff4a74ddb84e service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:42:54.557758 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 709/1425] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:42:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:42:55.478246 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cafaa7-d8c2-4ba7-97ee-fe8a6eb65f54 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:55.478805 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cafaa7-d8c2-4ba7-97ee-fe8a6eb65f54 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:55.478805 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cafaa7-d8c2-4ba7-97ee-fe8a6eb65f54 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:55.480156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 19:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:55.485213 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e91d87-58b0-4f69-8bdc-0d23d1ea93fa None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:55.485670 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e91d87-58b0-4f69-8bdc-0d23d1ea93fa None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:55.485867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e91d87-58b0-4f69-8bdc-0d23d1ea93fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:55.503711 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-42e91d87-58b0-4f69-8bdc-0d23d1ea93fa None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:55.504882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-42e91d87-58b0-4f69-8bdc-0d23d1ea93fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:55.512553 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 19:42:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:55.666905 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cafaa7-d8c2-4ba7-97ee-fe8a6eb65f54 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:42:55.682240 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cafaa7-d8c2-4ba7-97ee-fe8a6eb65f54 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:42:55.688283 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97e9c5-2d6f-4331-8019-e51725e390fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:42:55.688918 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97e9c5-2d6f-4331-8019-e51725e390fd service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:55.689230 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97e9c5-2d6f-4331-8019-e51725e390fd service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:55.689507 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97e9c5-2d6f-4331-8019-e51725e390fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:55.697479 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97e9c5-2d6f-4331-8019-e51725e390fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:42:55.699057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff97e9c5-2d6f-4331-8019-e51725e390fd service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:42:55.708294 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 718/1428] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:42:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:42:56.635833 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161c4dc-d08d-42c6-880b-f0fa8f61f3e5 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:56.636391 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161c4dc-d08d-42c6-880b-f0fa8f61f3e5 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:56.636391 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9161c4dc-d08d-42c6-880b-f0fa8f61f3e5 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:56.637907 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 19:42:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:56.642793 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effe750-e17a-47a3-8d39-05db999315bb None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:56.643242 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effe750-e17a-47a3-8d39-05db999315bb None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:56.643385 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effe750-e17a-47a3-8d39-05db999315bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:56.661141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-7effe750-e17a-47a3-8d39-05db999315bb None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:56.662321 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-7effe750-e17a-47a3-8d39-05db999315bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:56.670478 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 19:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:56.820870 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161c4dc-d08d-42c6-880b-f0fa8f61f3e5 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:42:56.835874 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9161c4dc-d08d-42c6-880b-f0fa8f61f3e5 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:42:56.842234 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce018b7c-54cd-4e0b-9501-eb1eb6ec1f2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:42:56.842900 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce018b7c-54cd-4e0b-9501-eb1eb6ec1f2f service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:56.843183 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce018b7c-54cd-4e0b-9501-eb1eb6ec1f2f service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:56.843458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce018b7c-54cd-4e0b-9501-eb1eb6ec1f2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:56.851640 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce018b7c-54cd-4e0b-9501-eb1eb6ec1f2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:42:56.853292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce018b7c-54cd-4e0b-9501-eb1eb6ec1f2f service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:42:56.863292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 712/1431] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:42:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:42:57.777400 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fa665-8bfa-4d50-a023-29ffa6a5f35c None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:57.777923 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fa665-8bfa-4d50-a023-29ffa6a5f35c None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:57.777923 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6fa665-8bfa-4d50-a023-29ffa6a5f35c None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:57.779825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 19:42:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:57.786032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75279dd4-f7e0-45cf-8a1e-a8889695e6d8 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:57.786485 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75279dd4-f7e0-45cf-8a1e-a8889695e6d8 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:57.786799 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75279dd4-f7e0-45cf-8a1e-a8889695e6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:57.805333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-75279dd4-f7e0-45cf-8a1e-a8889695e6d8 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:57.806509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-75279dd4-f7e0-45cf-8a1e-a8889695e6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:57.814643 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 19:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:57.967122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fa665-8bfa-4d50-a023-29ffa6a5f35c None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:42:57.981075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6fa665-8bfa-4d50-a023-29ffa6a5f35c None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:42:57.986354 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dee3ad-bab3-45a4-8507-44eb03c6202f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:42:57.986987 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dee3ad-bab3-45a4-8507-44eb03c6202f service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:57.987269 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dee3ad-bab3-45a4-8507-44eb03c6202f service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:57.987544 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dee3ad-bab3-45a4-8507-44eb03c6202f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:57.995474 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dee3ad-bab3-45a4-8507-44eb03c6202f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:42:57.997203 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dee3ad-bab3-45a4-8507-44eb03c6202f service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:42:58.006304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 721/1434] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:42:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:42:58.930744 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f108e-67b4-4bed-b155-b4eb2e16effb None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:58.931266 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f108e-67b4-4bed-b155-b4eb2e16effb None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:58.931266 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f108e-67b4-4bed-b155-b4eb2e16effb None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:58.932588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 19:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:42:58.938460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cb255-9b62-480f-b1a0-f55a46223ff1 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:58.938881 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cb255-9b62-480f-b1a0-f55a46223ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:58.939021 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cb255-9b62-480f-b1a0-f55a46223ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:58.956584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-973cb255-9b62-480f-b1a0-f55a46223ff1 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:42:58.957802 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-973cb255-9b62-480f-b1a0-f55a46223ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:42:58.965992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 19:42:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:42:59.118836 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f108e-67b4-4bed-b155-b4eb2e16effb None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:42:59.133407 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f108e-67b4-4bed-b155-b4eb2e16effb None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:42:59.139448 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93260d1-08c9-4d80-bf2b-4c9eb0bd798e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:42:59.140085 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93260d1-08c9-4d80-bf2b-4c9eb0bd798e service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:42:59.140372 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93260d1-08c9-4d80-bf2b-4c9eb0bd798e service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:42:59.140651 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93260d1-08c9-4d80-bf2b-4c9eb0bd798e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:42:59.148581 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93260d1-08c9-4d80-bf2b-4c9eb0bd798e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:42:59.150202 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93260d1-08c9-4d80-bf2b-4c9eb0bd798e service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:42:59.159571 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 715/1437] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:42:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:43:00.100632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e76458-6071-4afc-8b51-068ad8722f8b None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:00.101248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e76458-6071-4afc-8b51-068ad8722f8b None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:00.101248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e76458-6071-4afc-8b51-068ad8722f8b None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:00.102591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 19:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:43:00.108123 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda504e-1b76-4859-a8a8-3912f9cbeab7 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:00.108533 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda504e-1b76-4859-a8a8-3912f9cbeab7 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:00.108682 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda504e-1b76-4859-a8a8-3912f9cbeab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:00.127184 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-bcda504e-1b76-4859-a8a8-3912f9cbeab7 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:43:00.128369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-bcda504e-1b76-4859-a8a8-3912f9cbeab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:43:00.135965 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 19:43:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:43:00.284637 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e76458-6071-4afc-8b51-068ad8722f8b None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:43:00.299516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e76458-6071-4afc-8b51-068ad8722f8b None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:43:00.304990 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2bb60a-78e7-4bab-a469-12b4516f4421 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:43:00.305629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bb60a-78e7-4bab-a469-12b4516f4421 service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:00.305914 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bb60a-78e7-4bab-a469-12b4516f4421 service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:00.306189 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2bb60a-78e7-4bab-a469-12b4516f4421 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:00.315055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2bb60a-78e7-4bab-a469-12b4516f4421 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:43:00.320918 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2bb60a-78e7-4bab-a469-12b4516f4421 service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:43:00.325955 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 724/1440] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:43:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:43:02.467458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a00ecc-afa3-4f3a-b8ce-c9dd2cfff412 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:02.467994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a00ecc-afa3-4f3a-b8ce-c9dd2cfff412 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:02.467994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a00ecc-afa3-4f3a-b8ce-c9dd2cfff412 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:02.469436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 19:43:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:43:02.474854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1539838-4a1d-49b9-825a-818d4efe41f9 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:02.475247 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1539838-4a1d-49b9-825a-818d4efe41f9 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:02.475418 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1539838-4a1d-49b9-825a-818d4efe41f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:02.493105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-c1539838-4a1d-49b9-825a-818d4efe41f9 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:43:02.494416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-c1539838-4a1d-49b9-825a-818d4efe41f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:43:02.502590 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 19:43:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:43:02.541983 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a00ecc-afa3-4f3a-b8ce-c9dd2cfff412 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:43:02.552738 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a00ecc-afa3-4f3a-b8ce-c9dd2cfff412 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:43:02.556194 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9a822c-67ba-4f49-a2a1-7db6a5ad1a51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:43:02.556826 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a822c-67ba-4f49-a2a1-7db6a5ad1a51 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:02.557148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a822c-67ba-4f49-a2a1-7db6a5ad1a51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:02.557426 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9a822c-67ba-4f49-a2a1-7db6a5ad1a51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:02.566206 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9a822c-67ba-4f49-a2a1-7db6a5ad1a51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:43:02.567625 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9a822c-67ba-4f49-a2a1-7db6a5ad1a51 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:43:02.577259 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 718/1443] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:43:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:43:05.008495 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348d613-daf4-4749-876e-e01dbc709a07 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:05.009102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348d613-daf4-4749-876e-e01dbc709a07 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:05.009215 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a348d613-daf4-4749-876e-e01dbc709a07 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:05.010579 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 19:43:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:43:05.016244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212564e4-0008-4085-b192-b9f63010cbef None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:05.016684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212564e4-0008-4085-b192-b9f63010cbef None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:05.016801 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212564e4-0008-4085-b192-b9f63010cbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:05.034910 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-212564e4-0008-4085-b192-b9f63010cbef None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:43:05.036097 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-212564e4-0008-4085-b192-b9f63010cbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:43:05.044671 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 19:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:43:05.083582 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348d613-daf4-4749-876e-e01dbc709a07 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:43:05.095240 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a348d613-daf4-4749-876e-e01dbc709a07 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:43:05.098783 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd712dcc-c5a9-4c2e-88c2-fd2e6846dae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:43:05.099417 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd712dcc-c5a9-4c2e-88c2-fd2e6846dae8 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:05.099701 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd712dcc-c5a9-4c2e-88c2-fd2e6846dae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:05.099976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd712dcc-c5a9-4c2e-88c2-fd2e6846dae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:05.109475 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd712dcc-c5a9-4c2e-88c2-fd2e6846dae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:43:05.110909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd712dcc-c5a9-4c2e-88c2-fd2e6846dae8 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:43:05.120068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 727/1446] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:43:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:43:07.678935 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207341a-6c5e-4125-b9cf-33674668e10c None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:07.679601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207341a-6c5e-4125-b9cf-33674668e10c None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:07.679601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207341a-6c5e-4125-b9cf-33674668e10c None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:07.680872 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 19:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:43:07.686307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efba7cd-3804-4109-af1c-b721c5a2dbab None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:07.686775 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efba7cd-3804-4109-af1c-b721c5a2dbab None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:07.686881 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efba7cd-3804-4109-af1c-b721c5a2dbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:07.704560 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-1efba7cd-3804-4109-af1c-b721c5a2dbab None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:43:07.705765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-1efba7cd-3804-4109-af1c-b721c5a2dbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:43:07.713308 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 19:43:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:43:07.752035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207341a-6c5e-4125-b9cf-33674668e10c None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:43:07.763070 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207341a-6c5e-4125-b9cf-33674668e10c None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:43:07.766532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1193af0-8e02-442d-9c39-8b66f6b8f714 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:43:07.767168 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1193af0-8e02-442d-9c39-8b66f6b8f714 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:07.767455 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1193af0-8e02-442d-9c39-8b66f6b8f714 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:07.767736 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1193af0-8e02-442d-9c39-8b66f6b8f714 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:07.776431 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1193af0-8e02-442d-9c39-8b66f6b8f714 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:43:07.778435 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1193af0-8e02-442d-9c39-8b66f6b8f714 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:43:07.787567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 721/1449] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:43:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:43:10.113627 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85909761-c909-4459-9504-afdb19b6c074 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:10.114198 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85909761-c909-4459-9504-afdb19b6c074 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:10.114198 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85909761-c909-4459-9504-afdb19b6c074 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:10.115540 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 19:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:43:10.120968 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338dfab-5c51-4917-81dd-03d67a6eed6e None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:10.121402 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338dfab-5c51-4917-81dd-03d67a6eed6e None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:10.121649 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b338dfab-5c51-4917-81dd-03d67a6eed6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:10.139811 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-b338dfab-5c51-4917-81dd-03d67a6eed6e None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:43:10.141043 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-b338dfab-5c51-4917-81dd-03d67a6eed6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:43:10.148707 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 19:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:43:10.188426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85909761-c909-4459-9504-afdb19b6c074 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:43:10.199892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85909761-c909-4459-9504-afdb19b6c074 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:43:10.203629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d089219-16d3-4949-88cc-3c67aa7c6b16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:43:10.212521 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d089219-16d3-4949-88cc-3c67aa7c6b16 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:10.212521 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d089219-16d3-4949-88cc-3c67aa7c6b16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:10.212521 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d089219-16d3-4949-88cc-3c67aa7c6b16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:10.214744 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d089219-16d3-4949-88cc-3c67aa7c6b16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:43:10.220133 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d089219-16d3-4949-88cc-3c67aa7c6b16 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:43:10.231034 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 730/1452] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:43:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:43:12.620035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a1391-1c74-400e-9a46-cf106133c9f5 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:12.621218 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a1391-1c74-400e-9a46-cf106133c9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:12.621218 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41a1391-1c74-400e-9a46-cf106133c9f5 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:12.622568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 19:43:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:43:12.628189 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6036d02-c00f-4bf9-8178-1d3b77b08ab3 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:12.628654 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6036d02-c00f-4bf9-8178-1d3b77b08ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:12.628760 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6036d02-c00f-4bf9-8178-1d3b77b08ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:12.647559 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-c6036d02-c00f-4bf9-8178-1d3b77b08ab3 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:43:12.648823 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-c6036d02-c00f-4bf9-8178-1d3b77b08ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:43:12.657784 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 19:43:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:43:12.695941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a1391-1c74-400e-9a46-cf106133c9f5 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:43:12.707286 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41a1391-1c74-400e-9a46-cf106133c9f5 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:43:12.710784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4eb21d-123f-496f-9982-535b245aca8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:43:12.711419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4eb21d-123f-496f-9982-535b245aca8b service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:12.711708 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4eb21d-123f-496f-9982-535b245aca8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:12.711986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4eb21d-123f-496f-9982-535b245aca8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:12.721495 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4eb21d-123f-496f-9982-535b245aca8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:43:12.722929 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4eb21d-123f-496f-9982-535b245aca8b service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:43:12.732088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 724/1455] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:43:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:43:15.025291 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9805f6-a9be-45d5-ae5a-d9279904e7d7 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:15.025866 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9805f6-a9be-45d5-ae5a-d9279904e7d7 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:15.025866 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9805f6-a9be-45d5-ae5a-d9279904e7d7 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:15.027710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 19:43:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:43:15.033173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de508145-2a9c-49af-a68f-8a23b2e8ba6b None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:15.033601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de508145-2a9c-49af-a68f-8a23b2e8ba6b None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:15.033780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de508145-2a9c-49af-a68f-8a23b2e8ba6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:15.053646 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-de508145-2a9c-49af-a68f-8a23b2e8ba6b None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:43:15.054914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-de508145-2a9c-49af-a68f-8a23b2e8ba6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:43:15.064432 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 19:43:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:43:15.102762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9805f6-a9be-45d5-ae5a-d9279904e7d7 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:43:15.113474 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9805f6-a9be-45d5-ae5a-d9279904e7d7 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:43:15.116858 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd97b34a-cb3c-40d5-9b14-12bdd8212a4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:43:15.117511 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97b34a-cb3c-40d5-9b14-12bdd8212a4f service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:15.117793 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97b34a-cb3c-40d5-9b14-12bdd8212a4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:15.118067 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd97b34a-cb3c-40d5-9b14-12bdd8212a4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:15.126960 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97b34a-cb3c-40d5-9b14-12bdd8212a4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:43:15.128378 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd97b34a-cb3c-40d5-9b14-12bdd8212a4f service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:43:15.138160 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 733/1458] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:43:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:43:17.411775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0570add-6db1-416d-94b3-63693f2f66a7 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:17.412312 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0570add-6db1-416d-94b3-63693f2f66a7 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:17.412312 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0570add-6db1-416d-94b3-63693f2f66a7 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:17.413662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 19:43:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:43:17.419746 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d7c3f-35f5-4720-92d3-7d946518408a None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:17.420160 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d7c3f-35f5-4720-92d3-7d946518408a None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:17.420297 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d7c3f-35f5-4720-92d3-7d946518408a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:17.426213 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=70 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:43:17.438518 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-a11d7c3f-35f5-4720-92d3-7d946518408a None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:43:17.439689 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-a11d7c3f-35f5-4720-92d3-7d946518408a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:43:17.447311 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 19:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:43:17.485224 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0570add-6db1-416d-94b3-63693f2f66a7 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:43:17.490533 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=73 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:43:17.496825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0570add-6db1-416d-94b3-63693f2f66a7 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:43:17.500585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b74657-606a-48b4-a1f3-3e034f9aabbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:43:17.500929 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b74657-606a-48b4-a1f3-3e034f9aabbf service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:17.501241 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b74657-606a-48b4-a1f3-3e034f9aabbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:17.501523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b74657-606a-48b4-a1f3-3e034f9aabbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:17.510622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b74657-606a-48b4-a1f3-3e034f9aabbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:43:17.512146 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b74657-606a-48b4-a1f3-3e034f9aabbf service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:43:17.522529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 727/1461] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:43:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:43:19.766998 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93bdb2-fa11-436d-ae8a-b0731047b0db None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:19.767597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93bdb2-fa11-436d-ae8a-b0731047b0db None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:19.767597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc93bdb2-fa11-436d-ae8a-b0731047b0db None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:19.768816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 19:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:43:19.774223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d41c0-38dc-45a3-9405-3061c665a016 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:19.774650 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d41c0-38dc-45a3-9405-3061c665a016 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:19.774785 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d41c0-38dc-45a3-9405-3061c665a016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:19.792698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-1b3d41c0-38dc-45a3-9405-3061c665a016 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:43:19.793939 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-1b3d41c0-38dc-45a3-9405-3061c665a016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:43:19.801659 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 19:43:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:43:19.839389 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93bdb2-fa11-436d-ae8a-b0731047b0db None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:43:19.841330 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-fc93bdb2-fa11-436d-ae8a-b0731047b0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:43:19.853800 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc93bdb2-fa11-436d-ae8a-b0731047b0db None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:43:19.857432 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb77748-8fa2-4b02-99d7-abdcade6c660 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:43:19.858058 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb77748-8fa2-4b02-99d7-abdcade6c660 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:19.858336 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb77748-8fa2-4b02-99d7-abdcade6c660 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:19.858606 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb77748-8fa2-4b02-99d7-abdcade6c660 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:19.867189 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb77748-8fa2-4b02-99d7-abdcade6c660 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:43:19.868594 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb77748-8fa2-4b02-99d7-abdcade6c660 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:43:19.877747 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 736/1464] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:43:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:43:22.302427 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b300db2-8b9a-4ac9-abbb-7274af538028 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:22.303005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b300db2-8b9a-4ac9-abbb-7274af538028 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:22.303005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b300db2-8b9a-4ac9-abbb-7274af538028 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:22.304486 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 19:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:43:22.310341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed722c83-8981-4e61-b57b-7e4957dad6f2 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:22.310798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed722c83-8981-4e61-b57b-7e4957dad6f2 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:22.310905 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed722c83-8981-4e61-b57b-7e4957dad6f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:22.328995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-ed722c83-8981-4e61-b57b-7e4957dad6f2 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:43:22.330171 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ed722c83-8981-4e61-b57b-7e4957dad6f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:43:22.337843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 19:43:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:43:22.374899 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b300db2-8b9a-4ac9-abbb-7274af538028 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:43:22.385468 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b300db2-8b9a-4ac9-abbb-7274af538028 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:43:22.388883 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db5357-0ba3-41f4-b2b3-d1947ddfb979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:43:22.389549 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db5357-0ba3-41f4-b2b3-d1947ddfb979 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:22.389832 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db5357-0ba3-41f4-b2b3-d1947ddfb979 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:22.390106 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db5357-0ba3-41f4-b2b3-d1947ddfb979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:22.398820 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db5357-0ba3-41f4-b2b3-d1947ddfb979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:43:22.400249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db5357-0ba3-41f4-b2b3-d1947ddfb979 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:43:22.410228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 730/1467] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:43:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:43:24.665660 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a4767-f87f-41b1-a925-89df69019048 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:24.666762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a4767-f87f-41b1-a925-89df69019048 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:24.666858 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02a4767-f87f-41b1-a925-89df69019048 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:24.668220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 19:43:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:43:24.673498 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe678b8-b6f4-4b9b-8383-f05a052fed3c None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:24.673931 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe678b8-b6f4-4b9b-8383-f05a052fed3c None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:24.674057 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe678b8-b6f4-4b9b-8383-f05a052fed3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:24.691818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-ebe678b8-b6f4-4b9b-8383-f05a052fed3c None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:43:24.693044 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-ebe678b8-b6f4-4b9b-8383-f05a052fed3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:43:24.700742 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 19:43:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:43:24.737824 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a4767-f87f-41b1-a925-89df69019048 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:43:24.749239 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02a4767-f87f-41b1-a925-89df69019048 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:43:24.752738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd54bc-e91c-4928-8f8b-7aab7e940cbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:43:24.753408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd54bc-e91c-4928-8f8b-7aab7e940cbc service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:24.753697 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd54bc-e91c-4928-8f8b-7aab7e940cbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:24.753977 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd54bc-e91c-4928-8f8b-7aab7e940cbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:24.763286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd54bc-e91c-4928-8f8b-7aab7e940cbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:43:24.764662 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fd54bc-e91c-4928-8f8b-7aab7e940cbc service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:43:24.774207 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 739/1470] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:43:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:43:34.686734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=17 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:43:34.770229 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=17 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:43:57.926666 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a415230-30d9-4de7-9d19-ee3c84e20004 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:57.927286 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a415230-30d9-4de7-9d19-ee3c84e20004 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:57.927286 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a415230-30d9-4de7-9d19-ee3c84e20004 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:57.929289 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 732/1471] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:43:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:43:57.935070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb2b27-9012-48d6-b6d0-053b46405e9a None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:57.935537 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb2b27-9012-48d6-b6d0-053b46405e9a None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:57.935658 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bb2b27-9012-48d6-b6d0-053b46405e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:57.953516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-e8bb2b27-9012-48d6-b6d0-053b46405e9a None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:43:57.954707 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-e8bb2b27-9012-48d6-b6d0-053b46405e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:43:57.962984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 740/1472] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:43:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:43:58.011122 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1100ac-0475-4db5-b613-46c643410190 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:58.011412 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1100ac-0475-4db5-b613-46c643410190 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:58.011694 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1100ac-0475-4db5-b613-46c643410190 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:58.013083 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 733/1473] 10.222.0.13 () {54 vars in 962 bytes} [Fri May 27 19:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:43:58.017744 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10efb4-503c-4b15-8348-8f2e87b79bbb None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:58.018064 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10efb4-503c-4b15-8348-8f2e87b79bbb None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:58.018345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10efb4-503c-4b15-8348-8f2e87b79bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:58.035907 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-eb10efb4-503c-4b15-8348-8f2e87b79bbb None None] MFA Rules not processed for user `1347c0bd4ada4139ad194d423958be6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:43:58.037158 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-eb10efb4-503c-4b15-8348-8f2e87b79bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:43:58.044098 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 741/1474] 10.222.0.13 () {58 vars in 1059 bytes} [Fri May 27 19:43:58 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:43:58.051402 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1100ac-0475-4db5-b613-46c643410190 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:43:58.062810 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1100ac-0475-4db5-b613-46c643410190 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:43:58.066361 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f7da9-8446-425e-a794-0e52d93752db service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1347c0bd4ada4139ad194d423958be6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:43:58.066992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f7da9-8446-425e-a794-0e52d93752db service glance] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:43:58.067277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f7da9-8446-425e-a794-0e52d93752db service glance] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:43:58.067556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f7da9-8446-425e-a794-0e52d93752db service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:43:58.076344 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f7da9-8446-425e-a794-0e52d93752db service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:43:58.077800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f7da9-8446-425e-a794-0e52d93752db service glance] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:43:58.086930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 734/1475] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:43:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:44:00.384664 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960c7be-1497-412f-a641-66f037f5bbd5 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:44:00.385277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960c7be-1497-412f-a641-66f037f5bbd5 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:44:00.385277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960c7be-1497-412f-a641-66f037f5bbd5 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:44:00.386617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 742/1476] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:44:00.391842 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3a988-910c-4233-a102-cde03771a95e None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:44:00.392280 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3a988-910c-4233-a102-cde03771a95e None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:44:00.392401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3a988-910c-4233-a102-cde03771a95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:44:00.410174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-2ac3a988-910c-4233-a102-cde03771a95e None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:44:00.411364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-2ac3a988-910c-4233-a102-cde03771a95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:44:00.419718 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 735/1477] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:44:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:44:00.442308 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960c7be-1497-412f-a641-66f037f5bbd5 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:44:00.454565 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960c7be-1497-412f-a641-66f037f5bbd5 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:44:00.459003 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ac6e66-6e9d-40f7-95eb-ce0785b74459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:44:00.459648 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac6e66-6e9d-40f7-95eb-ce0785b74459 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:44:00.459930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac6e66-6e9d-40f7-95eb-ce0785b74459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:44:00.460208 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ac6e66-6e9d-40f7-95eb-ce0785b74459 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:44:00.461536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac6e66-6e9d-40f7-95eb-ce0785b74459 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:44:00.463496 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: WARNING py.warnings [None req-b8ac6e66-6e9d-40f7-95eb-ce0785b74459 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 19:44:00.463496 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: warnings.warn(msg) May 27 19:44:00.463496 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: May 27 19:44:00.464630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ac6e66-6e9d-40f7-95eb-ce0785b74459 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:44:00.484387 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 743/1478] 10.222.0.13 () {56 vars in 1108 bytes} [Fri May 27 19:44:00 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:44:02.708563 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ea7a7-4f07-4290-9990-dfc368f6de52 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:44:02.709143 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ea7a7-4f07-4290-9990-dfc368f6de52 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:44:02.709230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ea7a7-4f07-4290-9990-dfc368f6de52 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:44:02.711136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 736/1479] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:44:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:44:02.716839 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381489f5-9dc7-4d82-89ae-b37edfde22d2 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:44:02.717273 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381489f5-9dc7-4d82-89ae-b37edfde22d2 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:44:02.717457 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381489f5-9dc7-4d82-89ae-b37edfde22d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:44:02.735355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-381489f5-9dc7-4d82-89ae-b37edfde22d2 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:44:02.736541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-381489f5-9dc7-4d82-89ae-b37edfde22d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:44:02.744289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 744/1480] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:44:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:44:02.787177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ea7a7-4f07-4290-9990-dfc368f6de52 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:44:02.802484 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ea7a7-4f07-4290-9990-dfc368f6de52 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:44:02.808489 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f81dfa-031c-4be4-a25c-a08b246b2c0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:44:02.809180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f81dfa-031c-4be4-a25c-a08b246b2c0e service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:44:02.809470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f81dfa-031c-4be4-a25c-a08b246b2c0e service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:44:02.809751 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f81dfa-031c-4be4-a25c-a08b246b2c0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:44:02.818183 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f81dfa-031c-4be4-a25c-a08b246b2c0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:44:02.819800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f81dfa-031c-4be4-a25c-a08b246b2c0e service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:44:02.829079 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 737/1481] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:44:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:44:05.212459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d58a4b-9c69-4e19-8d5c-5531119ccdf9 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:44:05.213299 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d58a4b-9c69-4e19-8d5c-5531119ccdf9 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:44:05.213299 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d58a4b-9c69-4e19-8d5c-5531119ccdf9 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:44:05.214659 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 745/1482] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:44:05.219876 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de0310-ef1a-4bee-a3d4-a67ad1d65cf1 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:44:05.220308 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de0310-ef1a-4bee-a3d4-a67ad1d65cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:44:05.220445 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de0310-ef1a-4bee-a3d4-a67ad1d65cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:44:05.238382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-d9de0310-ef1a-4bee-a3d4-a67ad1d65cf1 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:44:05.239572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d9de0310-ef1a-4bee-a3d4-a67ad1d65cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:44:05.248171 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 738/1483] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:44:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:44:05.283615 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d58a4b-9c69-4e19-8d5c-5531119ccdf9 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:44:05.294956 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d58a4b-9c69-4e19-8d5c-5531119ccdf9 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:44:05.298396 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b087e-3bc5-446b-a507-25ef51c831f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:44:05.299030 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b087e-3bc5-446b-a507-25ef51c831f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:44:05.299312 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b087e-3bc5-446b-a507-25ef51c831f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:44:05.299587 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b087e-3bc5-446b-a507-25ef51c831f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:44:05.308831 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652b087e-3bc5-446b-a507-25ef51c831f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:44:05.310304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652b087e-3bc5-446b-a507-25ef51c831f8 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:44:05.319455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 746/1484] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:44:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:44:07.578445 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248593d4-ab11-47a1-a306-e423b9403b9f None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:44:07.579005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248593d4-ab11-47a1-a306-e423b9403b9f None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:44:07.579285 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248593d4-ab11-47a1-a306-e423b9403b9f None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:44:07.580682 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 739/1485] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:44:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:44:07.585950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a915019-6e1a-4727-8004-236afb5564fd None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:44:07.586879 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a915019-6e1a-4727-8004-236afb5564fd None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:44:07.587145 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a915019-6e1a-4727-8004-236afb5564fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:44:07.604541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-9a915019-6e1a-4727-8004-236afb5564fd None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:44:07.605750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-9a915019-6e1a-4727-8004-236afb5564fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:44:07.613793 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 747/1486] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:44:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:44:07.653559 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248593d4-ab11-47a1-a306-e423b9403b9f None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:44:07.664513 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248593d4-ab11-47a1-a306-e423b9403b9f None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:44:07.668067 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7b57fe-6127-4a6e-83c7-07cc1b087fbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:44:07.668700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b57fe-6127-4a6e-83c7-07cc1b087fbe service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:44:07.669013 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b57fe-6127-4a6e-83c7-07cc1b087fbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:44:07.669288 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7b57fe-6127-4a6e-83c7-07cc1b087fbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:44:07.677926 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b57fe-6127-4a6e-83c7-07cc1b087fbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:44:07.679350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7b57fe-6127-4a6e-83c7-07cc1b087fbe service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:44:07.688386 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 740/1487] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:44:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:44:09.975717 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77410bd1-8749-43d8-b343-eea6a8c510b0 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:44:09.976315 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77410bd1-8749-43d8-b343-eea6a8c510b0 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:44:09.976315 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77410bd1-8749-43d8-b343-eea6a8c510b0 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:44:09.977810 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 748/1488] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:44:09.983104 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f016c74-87c8-49ee-b801-41c892cc9575 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:44:09.983550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f016c74-87c8-49ee-b801-41c892cc9575 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:44:09.983656 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f016c74-87c8-49ee-b801-41c892cc9575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:44:10.000853 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-4f016c74-87c8-49ee-b801-41c892cc9575 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:44:10.002052 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-4f016c74-87c8-49ee-b801-41c892cc9575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:44:10.010277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 741/1489] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:44:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:44:10.051266 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77410bd1-8749-43d8-b343-eea6a8c510b0 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:44:10.061753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77410bd1-8749-43d8-b343-eea6a8c510b0 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:44:10.065206 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793e6c0a-7d43-405b-88b7-2e806c712485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:44:10.065848 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e6c0a-7d43-405b-88b7-2e806c712485 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:44:10.066129 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e6c0a-7d43-405b-88b7-2e806c712485 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:44:10.066402 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793e6c0a-7d43-405b-88b7-2e806c712485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:44:10.075747 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e6c0a-7d43-405b-88b7-2e806c712485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:44:10.077214 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793e6c0a-7d43-405b-88b7-2e806c712485 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:44:10.087041 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 749/1490] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:44:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:44:12.775100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd1466-9b81-4f77-a460-896f5077a2c9 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:44:12.775632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd1466-9b81-4f77-a460-896f5077a2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:44:12.775632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd1466-9b81-4f77-a460-896f5077a2c9 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:44:12.777347 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 742/1491] 10.222.0.13 () {54 vars in 926 bytes} [Fri May 27 19:44:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:44:12.782856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec59d4-4c81-4506-b4c0-2cb2fd67b688 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:44:12.783296 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec59d4-4c81-4506-b4c0-2cb2fd67b688 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:44:12.783414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec59d4-4c81-4506-b4c0-2cb2fd67b688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:44:12.800939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-95ec59d4-4c81-4506-b4c0-2cb2fd67b688 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:44:12.802148 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-95ec59d4-4c81-4506-b4c0-2cb2fd67b688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:44:12.810533 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 750/1492] 10.222.0.13 () {58 vars in 1023 bytes} [Fri May 27 19:44:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:44:12.854777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd1466-9b81-4f77-a460-896f5077a2c9 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:44:12.865804 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd1466-9b81-4f77-a460-896f5077a2c9 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:44:12.869672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a54ed0-ddc9-4a6e-9921-fb2b155f4a05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:44:12.870317 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a54ed0-ddc9-4a6e-9921-fb2b155f4a05 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:44:12.870600 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a54ed0-ddc9-4a6e-9921-fb2b155f4a05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:44:12.870874 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a54ed0-ddc9-4a6e-9921-fb2b155f4a05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:44:12.880434 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a54ed0-ddc9-4a6e-9921-fb2b155f4a05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:44:12.882011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a54ed0-ddc9-4a6e-9921-fb2b155f4a05 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:44:12.892240 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 743/1493] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:44:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:44:22.795956 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=33 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:44:22.887962 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=33 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:46:26.419708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd8b15-5692-4b6d-b779-f880512c3983 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:46:26.420275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd8b15-5692-4b6d-b779-f880512c3983 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:46:26.420275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd8b15-5692-4b6d-b779-f880512c3983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:46:26.438278 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-46dd8b15-5692-4b6d-b779-f880512c3983 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:46:26.440025 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-46dd8b15-5692-4b6d-b779-f880512c3983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:46:26.447602 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 751/1494] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:46:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:46:26.456870 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0271b3-b2b2-4bd6-8034-59146175f408 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:46:26.457398 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0271b3-b2b2-4bd6-8034-59146175f408 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:46:26.457501 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0271b3-b2b2-4bd6-8034-59146175f408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:46:26.475140 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-9a0271b3-b2b2-4bd6-8034-59146175f408 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:46:26.476322 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-9a0271b3-b2b2-4bd6-8034-59146175f408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:46:26.484043 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 744/1495] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:46:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:46:26.489247 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd8b15-5692-4b6d-b779-f880512c3983 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:46:26.498554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd8b15-5692-4b6d-b779-f880512c3983 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:46:26.503091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36f0b5-8c64-4f35-9976-c66e2f982488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:46:26.503734 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36f0b5-8c64-4f35-9976-c66e2f982488 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:46:26.504016 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36f0b5-8c64-4f35-9976-c66e2f982488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:46:26.504290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36f0b5-8c64-4f35-9976-c66e2f982488 admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:46:26.505041 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36f0b5-8c64-4f35-9976-c66e2f982488 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:46:26.505348 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36f0b5-8c64-4f35-9976-c66e2f982488 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:46:26.506369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: WARNING py.warnings [None req-8a36f0b5-8c64-4f35-9976-c66e2f982488 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 19:46:26.506369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: warnings.warn(msg) May 27 19:46:26.506369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: May 27 19:46:26.507248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36f0b5-8c64-4f35-9976-c66e2f982488 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:46:26.519062 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 752/1496] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:46:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:46:26.527199 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0271b3-b2b2-4bd6-8034-59146175f408 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:46:26.536847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0271b3-b2b2-4bd6-8034-59146175f408 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:46:26.541435 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031ba68-a619-4fcf-ac80-1c3328958a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:46:26.542090 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031ba68-a619-4fcf-ac80-1c3328958a15 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:46:26.542375 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031ba68-a619-4fcf-ac80-1c3328958a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:46:26.542659 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031ba68-a619-4fcf-ac80-1c3328958a15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:46:26.543351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031ba68-a619-4fcf-ac80-1c3328958a15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:46:26.545141 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: WARNING py.warnings [None req-9031ba68-a619-4fcf-ac80-1c3328958a15 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 19:46:26.545141 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: warnings.warn(msg) May 27 19:46:26.545141 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: May 27 19:46:26.546351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9031ba68-a619-4fcf-ac80-1c3328958a15 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:46:26.646335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 745/1497] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:46:26 2022] POST /identity/v3/projects => generated 377 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:46:26.650722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36f0b5-8c64-4f35-9976-c66e2f982488 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:46:26.661100 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36f0b5-8c64-4f35-9976-c66e2f982488 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:46:26.715689 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cd116-a0f2-4313-81cb-82ae1e80356c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:46:26.716351 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cd116-a0f2-4313-81cb-82ae1e80356c admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:46:26.716630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cd116-a0f2-4313-81cb-82ae1e80356c admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:46:26.716904 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cd116-a0f2-4313-81cb-82ae1e80356c admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:46:26.717605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9cd116-a0f2-4313-81cb-82ae1e80356c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:46:26.718619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: WARNING py.warnings [None req-ba9cd116-a0f2-4313-81cb-82ae1e80356c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 19:46:26.718619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: warnings.warn(msg) May 27 19:46:26.718619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: May 27 19:46:26.719812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9cd116-a0f2-4313-81cb-82ae1e80356c admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:46:26.802007 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 753/1498] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:46:26 2022] POST /identity/v3/users => generated 289 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:46:26.806101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031ba68-a619-4fcf-ac80-1c3328958a15 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:46:26.816502 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9031ba68-a619-4fcf-ac80-1c3328958a15 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:46:26.821066 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73607740-9085-4c4a-b301-19a6fcd92504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:46:26.821733 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73607740-9085-4c4a-b301-19a6fcd92504 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:46:26.822022 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73607740-9085-4c4a-b301-19a6fcd92504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:46:26.822303 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73607740-9085-4c4a-b301-19a6fcd92504 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:46:26.823066 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73607740-9085-4c4a-b301-19a6fcd92504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:46:26.824111 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: WARNING py.warnings [None req-73607740-9085-4c4a-b301-19a6fcd92504 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 19:46:26.824111 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: warnings.warn(msg) May 27 19:46:26.824111 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: May 27 19:46:26.825051 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73607740-9085-4c4a-b301-19a6fcd92504 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:46:26.838484 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 746/1499] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:46:26.842382 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cd116-a0f2-4313-81cb-82ae1e80356c admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:46:26.853880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9cd116-a0f2-4313-81cb-82ae1e80356c admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:46:26.859081 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4bb84-f9d8-4663-9dc8-dfb68d50bc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:46:26.859734 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4bb84-f9d8-4663-9dc8-dfb68d50bc37 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:46:26.860014 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4bb84-f9d8-4663-9dc8-dfb68d50bc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:46:26.860287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4bb84-f9d8-4663-9dc8-dfb68d50bc37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:46:26.861702 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4bb84-f9d8-4663-9dc8-dfb68d50bc37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:46:26.862710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: WARNING py.warnings [None req-1be4bb84-f9d8-4663-9dc8-dfb68d50bc37 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 19:46:26.862710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: warnings.warn(msg) May 27 19:46:26.862710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: May 27 19:46:26.863582 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4bb84-f9d8-4663-9dc8-dfb68d50bc37 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:46:26.876405 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 754/1500] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:46:26.880597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73607740-9085-4c4a-b301-19a6fcd92504 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:46:26.890679 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73607740-9085-4c4a-b301-19a6fcd92504 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:46:26.896006 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385fd3ba-23a0-421c-9e11-bd5cbb240f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:46:26.896935 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385fd3ba-23a0-421c-9e11-bd5cbb240f62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:46:26.897254 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385fd3ba-23a0-421c-9e11-bd5cbb240f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:46:26.897533 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385fd3ba-23a0-421c-9e11-bd5cbb240f62 admin admin] PATH_INFO: `/v3/projects/a9f1c431e41f442c9300358b730e13c4/users/6b6bf90f8742464e912f5a0d29cf048d/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:46:26.914986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385fd3ba-23a0-421c-9e11-bd5cbb240f62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9f1c431e41f442c9300358b730e13c4, user_id=6b6bf90f8742464e912f5a0d29cf048d, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:46:26.916034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: WARNING py.warnings [None req-385fd3ba-23a0-421c-9e11-bd5cbb240f62 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 19:46:26.916034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: warnings.warn(msg) May 27 19:46:26.916034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: May 27 19:46:26.918034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385fd3ba-23a0-421c-9e11-bd5cbb240f62 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:46:26.983652 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 747/1501] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:46:26 2022] PUT /identity/v3/projects/a9f1c431e41f442c9300358b730e13c4/users/6b6bf90f8742464e912f5a0d29cf048d/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:46:26.995586 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762c89a-11d3-4e56-9f60-90899dc575a2 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:46:26.995878 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762c89a-11d3-4e56-9f60-90899dc575a2 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:46:26.996160 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762c89a-11d3-4e56-9f60-90899dc575a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:46:27.022759 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-d762c89a-11d3-4e56-9f60-90899dc575a2 None None] MFA Rules not processed for user `6b6bf90f8742464e912f5a0d29cf048d`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:46:27.023973 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d762c89a-11d3-4e56-9f60-90899dc575a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:46:27.087887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 755/1502] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:46:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:46:27.101578 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0edc47-cd6a-4db0-aa94-f305384db219 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:46:27.101875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0edc47-cd6a-4db0-aa94-f305384db219 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:46:27.102158 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0edc47-cd6a-4db0-aa94-f305384db219 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:46:27.103532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 748/1503] 10.222.0.13 () {54 vars in 963 bytes} [Fri May 27 19:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:46:27.108812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0ca1d-4f4d-416d-968c-85c7a0fc7cfa None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:46:27.109160 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0ca1d-4f4d-416d-968c-85c7a0fc7cfa None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:46:27.109441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0ca1d-4f4d-416d-968c-85c7a0fc7cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:46:27.127215 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-ada0ca1d-4f4d-416d-968c-85c7a0fc7cfa None None] MFA Rules not processed for user `7b7e90ca75d643cbb0e6d6791fc30b45`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:46:27.128388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ada0ca1d-4f4d-416d-968c-85c7a0fc7cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:46:27.172085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:46:27.179560 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 756/1504] 10.222.0.13 () {58 vars in 1060 bytes} [Fri May 27 19:46:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:46:27.187318 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0edc47-cd6a-4db0-aa94-f305384db219 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:46:27.197894 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0edc47-cd6a-4db0-aa94-f305384db219 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:46:27.201394 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d2f47-44b4-4280-85b0-17f827461922 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:46:27.202029 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d2f47-44b4-4280-85b0-17f827461922 service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:46:27.202342 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d2f47-44b4-4280-85b0-17f827461922 service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:46:27.202621 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d2f47-44b4-4280-85b0-17f827461922 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:46:27.211651 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d2f47-44b4-4280-85b0-17f827461922 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:46:27.213111 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d2f47-44b4-4280-85b0-17f827461922 service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:46:27.223095 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 749/1505] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:46:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:46:27.264539 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9c254-f06d-4ef2-9c0b-5a141b0d3878 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:46:27.266870 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9c254-f06d-4ef2-9c0b-5a141b0d3878 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:46:27.266870 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d9c254-f06d-4ef2-9c0b-5a141b0d3878 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:46:27.267711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 757/1506] 10.222.0.13 () {52 vars in 829 bytes} [Fri May 27 19:46:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:46:27.271298 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d2f47-44b4-4280-85b0-17f827461922 service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:46:27.334320 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d2f47-44b4-4280-85b0-17f827461922 service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:46:27.338830 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f7e88-6dfa-4600-b36d-3ee4abb34d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:46:27.339699 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f7e88-6dfa-4600-b36d-3ee4abb34d08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:46:27.339979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f7e88-6dfa-4600-b36d-3ee4abb34d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:46:27.340252 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f7e88-6dfa-4600-b36d-3ee4abb34d08 admin admin] PATH_INFO: `/v3/users/6b6bf90f8742464e912f5a0d29cf048d` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:46:27.341503 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f7e88-6dfa-4600-b36d-3ee4abb34d08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b6bf90f8742464e912f5a0d29cf048d)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:46:27.342513 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: WARNING py.warnings [None req-a40f7e88-6dfa-4600-b36d-3ee4abb34d08 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 19:46:27.342513 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: warnings.warn(msg) May 27 19:46:27.342513 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: May 27 19:46:27.343707 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f7e88-6dfa-4600-b36d-3ee4abb34d08 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:46:27.777887 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-a40f7e88-6dfa-4600-b36d-3ee4abb34d08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b6bf90f8742464e912f5a0d29cf048d'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:46:27.796709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-a40f7e88-6dfa-4600-b36d-3ee4abb34d08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b6bf90f8742464e912f5a0d29cf048d'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:46:27.855881 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-a40f7e88-6dfa-4600-b36d-3ee4abb34d08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b6bf90f8742464e912f5a0d29cf048d'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:46:27.856580 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-a40f7e88-6dfa-4600-b36d-3ee4abb34d08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b6bf90f8742464e912f5a0d29cf048d'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:46:27.857207 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-a40f7e88-6dfa-4600-b36d-3ee4abb34d08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b6bf90f8742464e912f5a0d29cf048d'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:46:27.867768 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:46:27.877703 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 750/1507] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:46:27 2022] DELETE /identity/v3/users/6b6bf90f8742464e912f5a0d29cf048d => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:46:27.882176 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1531b26-7654-4beb-baff-d618e1549f3c None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:46:27.882652 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1531b26-7654-4beb-baff-d618e1549f3c None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:46:27.882750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1531b26-7654-4beb-baff-d618e1549f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:46:27.901266 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-b1531b26-7654-4beb-baff-d618e1549f3c None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:46:27.902449 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-b1531b26-7654-4beb-baff-d618e1549f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:46:27.960819 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 758/1508] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:46:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:46:27.971182 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f7e88-6dfa-4600-b36d-3ee4abb34d08 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:46:27.972584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-a40f7e88-6dfa-4600-b36d-3ee4abb34d08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:46:28.029339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f7e88-6dfa-4600-b36d-3ee4abb34d08 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:46:28.033037 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54abb2c8-4e43-49ec-82ee-4ffab0b8fc6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:46:28.033689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abb2c8-4e43-49ec-82ee-4ffab0b8fc6e service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:46:28.033977 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abb2c8-4e43-49ec-82ee-4ffab0b8fc6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:46:28.034256 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54abb2c8-4e43-49ec-82ee-4ffab0b8fc6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:46:28.042977 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54abb2c8-4e43-49ec-82ee-4ffab0b8fc6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:46:28.044413 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54abb2c8-4e43-49ec-82ee-4ffab0b8fc6e service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:46:28.053885 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 751/1509] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:46:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:46:28.538559 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1531b26-7654-4beb-baff-d618e1549f3c None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:46:28.539931 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-b1531b26-7654-4beb-baff-d618e1549f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:46:28.552766 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1531b26-7654-4beb-baff-d618e1549f3c None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:46:28.557600 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b4e41-3c51-4fbb-87d9-4efcf48a3347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:46:28.558469 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b4e41-3c51-4fbb-87d9-4efcf48a3347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:46:28.558783 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b4e41-3c51-4fbb-87d9-4efcf48a3347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:46:28.559017 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b4e41-3c51-4fbb-87d9-4efcf48a3347 admin admin] PATH_INFO: `/v3/projects/a9f1c431e41f442c9300358b730e13c4` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:46:28.560234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3b4e41-3c51-4fbb-87d9-4efcf48a3347 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9f1c431e41f442c9300358b730e13c4)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:46:28.561393 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: WARNING py.warnings [None req-dc3b4e41-3c51-4fbb-87d9-4efcf48a3347 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 19:46:28.561393 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: warnings.warn(msg) May 27 19:46:28.561393 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: May 27 19:46:28.565062 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3b4e41-3c51-4fbb-87d9-4efcf48a3347 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:46:28.718783 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-dc3b4e41-3c51-4fbb-87d9-4efcf48a3347 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9f1c431e41f442c9300358b730e13c4', 'request_id': 'req-dc3b4e41-3c51-4fbb-87d9-4efcf48a3347'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:46:28.725169 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-dc3b4e41-3c51-4fbb-87d9-4efcf48a3347 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9f1c431e41f442c9300358b730e13c4', 'request_id': 'req-dc3b4e41-3c51-4fbb-87d9-4efcf48a3347'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:46:28.739153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-dc3b4e41-3c51-4fbb-87d9-4efcf48a3347 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9f1c431e41f442c9300358b730e13c4', 'request_id': 'req-dc3b4e41-3c51-4fbb-87d9-4efcf48a3347'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:46:28.851208 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-dc3b4e41-3c51-4fbb-87d9-4efcf48a3347 admin admin] The token cache is being invalidate because project a9f1c431e41f442c9300358b730e13c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:46:28.851438 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-dc3b4e41-3c51-4fbb-87d9-4efcf48a3347 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:46:28.852051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-dc3b4e41-3c51-4fbb-87d9-4efcf48a3347 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:46:28.853768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 759/1510] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:46:28 2022] DELETE /identity/v3/projects/a9f1c431e41f442c9300358b730e13c4 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:46:38.049737 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=29 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:46:38.847810 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:48:10.103589 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b0bac-2633-4b9c-928c-73d42559ae60 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:10.104181 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b0bac-2633-4b9c-928c-73d42559ae60 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:10.104181 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b0bac-2633-4b9c-928c-73d42559ae60 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:10.105620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 19:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:48:10.110492 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f070b58-01cc-4ccc-8975-18aaf6ac5fbf None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:10.110932 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f070b58-01cc-4ccc-8975-18aaf6ac5fbf None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:10.111041 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f070b58-01cc-4ccc-8975-18aaf6ac5fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:10.129351 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-2f070b58-01cc-4ccc-8975-18aaf6ac5fbf None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:48:10.130554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-2f070b58-01cc-4ccc-8975-18aaf6ac5fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:48:10.188922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 19:48:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:48:10.362331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b0bac-2633-4b9c-928c-73d42559ae60 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:48:10.364014 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-333b0bac-2633-4b9c-928c-73d42559ae60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:48:10.438687 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b0bac-2633-4b9c-928c-73d42559ae60 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:48:10.444044 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23891d7e-0dc7-4d29-8fda-cefd4eb72425 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:48:10.445362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23891d7e-0dc7-4d29-8fda-cefd4eb72425 service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:10.445652 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23891d7e-0dc7-4d29-8fda-cefd4eb72425 service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:10.445935 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23891d7e-0dc7-4d29-8fda-cefd4eb72425 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:10.454278 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23891d7e-0dc7-4d29-8fda-cefd4eb72425 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:48:10.455874 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23891d7e-0dc7-4d29-8fda-cefd4eb72425 service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:48:10.466962 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 753/1513] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:48:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:48:10.572792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3c870-55d6-41a7-ba64-84c791f66463 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:10.573120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3c870-55d6-41a7-ba64-84c791f66463 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:10.573647 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3c870-55d6-41a7-ba64-84c791f66463 None None] PATH_INFO: `/v3` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:10.576126 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 761/1514] 10.222.0.17 () {54 vars in 925 bytes} [Fri May 27 19:48:10 2022] GET /identity/v3 => generated 255 bytes in 6 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:48:10.582986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b981f-25ee-42e4-93da-b4e985d7e5eb None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:10.583312 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b981f-25ee-42e4-93da-b4e985d7e5eb None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:10.583594 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b981f-25ee-42e4-93da-b4e985d7e5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:10.603535 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-627b981f-25ee-42e4-93da-b4e985d7e5eb None None] MFA Rules not processed for user `ffc2ed13a0c84f128b795002892efc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:48:10.604775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-627b981f-25ee-42e4-93da-b4e985d7e5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:48:10.670607 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 754/1515] 10.222.0.17 () {58 vars in 1015 bytes} [Fri May 27 19:48:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:48:11.351771 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640b411-cdb3-4c12-a5df-2438279aaa8b None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:11.352331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640b411-cdb3-4c12-a5df-2438279aaa8b None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:11.352331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640b411-cdb3-4c12-a5df-2438279aaa8b None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:11.353783 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 762/1516] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 19:48:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:48:11.358585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572b70e-cb71-4557-b32c-e44f478c5dc6 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:11.359034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572b70e-cb71-4557-b32c-e44f478c5dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:11.359135 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572b70e-cb71-4557-b32c-e44f478c5dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:11.377159 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-d572b70e-cb71-4557-b32c-e44f478c5dc6 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:48:11.378341 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d572b70e-cb71-4557-b32c-e44f478c5dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:48:11.386180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 755/1517] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 19:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:48:11.538930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640b411-cdb3-4c12-a5df-2438279aaa8b None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:48:11.553039 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640b411-cdb3-4c12-a5df-2438279aaa8b None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:48:11.558472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afd77d-5339-44e1-b7a1-835cc10aadfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:48:11.559108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afd77d-5339-44e1-b7a1-835cc10aadfb service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:11.559390 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afd77d-5339-44e1-b7a1-835cc10aadfb service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:11.559694 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afd77d-5339-44e1-b7a1-835cc10aadfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:11.567703 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6afd77d-5339-44e1-b7a1-835cc10aadfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:48:11.569857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6afd77d-5339-44e1-b7a1-835cc10aadfb service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:48:11.579142 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 763/1518] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:48:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:48:14.766736 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc674a65-d3c7-4ed0-b36b-69d7cff88544 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:14.767319 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc674a65-d3c7-4ed0-b36b-69d7cff88544 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:14.767319 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc674a65-d3c7-4ed0-b36b-69d7cff88544 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:14.768764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 756/1519] 10.222.0.13 () {54 vars in 960 bytes} [Fri May 27 19:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:48:14.773968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c016c-be73-46c4-8f5c-bf9fd8dfe5f3 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:14.774490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c016c-be73-46c4-8f5c-bf9fd8dfe5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:14.774598 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02c016c-be73-46c4-8f5c-bf9fd8dfe5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:14.792840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-f02c016c-be73-46c4-8f5c-bf9fd8dfe5f3 None None] MFA Rules not processed for user `ffc2ed13a0c84f128b795002892efc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:48:14.794069 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-f02c016c-be73-46c4-8f5c-bf9fd8dfe5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:48:14.802697 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 764/1520] 10.222.0.13 () {58 vars in 1057 bytes} [Fri May 27 19:48:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:48:14.810918 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc674a65-d3c7-4ed0-b36b-69d7cff88544 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:48:14.825061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc674a65-d3c7-4ed0-b36b-69d7cff88544 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:48:14.830874 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1954b86-81d0-4fa1-881f-4f88f5d73370 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2ed13a0c84f128b795002892efc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:48:14.831525 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1954b86-81d0-4fa1-881f-4f88f5d73370 service placement] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:14.831814 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1954b86-81d0-4fa1-881f-4f88f5d73370 service placement] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:14.832091 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1954b86-81d0-4fa1-881f-4f88f5d73370 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:14.841414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1954b86-81d0-4fa1-881f-4f88f5d73370 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:48:14.842714 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1954b86-81d0-4fa1-881f-4f88f5d73370 service placement] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:48:14.853268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 757/1521] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:48:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:48:22.867577 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d4433-b907-4143-9283-2a1bcf9e2185 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:22.868161 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d4433-b907-4143-9283-2a1bcf9e2185 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:22.868161 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22d4433-b907-4143-9283-2a1bcf9e2185 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:22.869499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 19:48:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:48:22.874356 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56913397-2f71-4667-8ddb-bd8742ce3be4 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:22.874808 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56913397-2f71-4667-8ddb-bd8742ce3be4 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:22.874907 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56913397-2f71-4667-8ddb-bd8742ce3be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:22.894708 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-56913397-2f71-4667-8ddb-bd8742ce3be4 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:48:22.895907 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-56913397-2f71-4667-8ddb-bd8742ce3be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:48:22.903824 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 19:48:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:48:23.062237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d4433-b907-4143-9283-2a1bcf9e2185 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:48:23.077271 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22d4433-b907-4143-9283-2a1bcf9e2185 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:48:23.082875 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4adfc-86b6-488a-8e6e-10129433b9fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:48:23.083508 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4adfc-86b6-488a-8e6e-10129433b9fa service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:23.083791 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4adfc-86b6-488a-8e6e-10129433b9fa service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:23.084065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4adfc-86b6-488a-8e6e-10129433b9fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:23.093251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e4adfc-86b6-488a-8e6e-10129433b9fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:48:23.094844 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e4adfc-86b6-488a-8e6e-10129433b9fa service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:48:23.104723 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 766/1524] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:48:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:48:23.983550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eec0bd-b46a-4cd2-9c13-ad6d2a303df0 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:23.984137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eec0bd-b46a-4cd2-9c13-ad6d2a303df0 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:23.984137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eec0bd-b46a-4cd2-9c13-ad6d2a303df0 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:23.985562 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 19:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:48:23.990419 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a9597-4429-4484-85e9-625c9df8f870 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:23.990844 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a9597-4429-4484-85e9-625c9df8f870 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:23.990982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68a9597-4429-4484-85e9-625c9df8f870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:24.008863 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-e68a9597-4429-4484-85e9-625c9df8f870 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:48:24.010065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-e68a9597-4429-4484-85e9-625c9df8f870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:48:24.017720 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 19:48:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:48:24.170101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eec0bd-b46a-4cd2-9c13-ad6d2a303df0 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:48:24.185734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eec0bd-b46a-4cd2-9c13-ad6d2a303df0 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:48:24.191278 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc84b6-0408-4286-9702-3ac02ac54e07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:48:24.191924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc84b6-0408-4286-9702-3ac02ac54e07 service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:24.192208 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc84b6-0408-4286-9702-3ac02ac54e07 service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:24.192488 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc84b6-0408-4286-9702-3ac02ac54e07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:24.200975 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc84b6-0408-4286-9702-3ac02ac54e07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:48:24.202599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bc84b6-0408-4286-9702-3ac02ac54e07 service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:48:24.212711 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 760/1527] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:48:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:48:28.642862 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c964a-29d4-40e4-a3cb-c627e3085622 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:28.643432 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c964a-29d4-40e4-a3cb-c627e3085622 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:28.643432 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c964a-29d4-40e4-a3cb-c627e3085622 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:28.644688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 19:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:48:28.650407 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecf686-c207-4fd7-b5e7-4209a61659a7 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:28.662413 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecf686-c207-4fd7-b5e7-4209a61659a7 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:28.662413 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aecf686-c207-4fd7-b5e7-4209a61659a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:28.669747 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-7aecf686-c207-4fd7-b5e7-4209a61659a7 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:48:28.670940 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-7aecf686-c207-4fd7-b5e7-4209a61659a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:48:28.678712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 19:48:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:48:28.716642 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c964a-29d4-40e4-a3cb-c627e3085622 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:48:28.718471 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-747c964a-29d4-40e4-a3cb-c627e3085622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:48:28.777117 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c964a-29d4-40e4-a3cb-c627e3085622 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:48:28.780747 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e76a244-f01e-4f4d-8e6f-ecb1392c9b81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:48:28.781437 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76a244-f01e-4f4d-8e6f-ecb1392c9b81 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:28.781723 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76a244-f01e-4f4d-8e6f-ecb1392c9b81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:28.782000 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e76a244-f01e-4f4d-8e6f-ecb1392c9b81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:28.792115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76a244-f01e-4f4d-8e6f-ecb1392c9b81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:48:28.793642 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e76a244-f01e-4f4d-8e6f-ecb1392c9b81 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:48:28.803283 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 769/1530] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:48:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:48:31.015658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745752c0-bcdd-4a7e-97de-4cf0a541c2ef None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:31.016225 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745752c0-bcdd-4a7e-97de-4cf0a541c2ef None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:31.016344 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745752c0-bcdd-4a7e-97de-4cf0a541c2ef None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:31.017672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 19:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:48:31.022864 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b942b-c0ae-4d4d-bcc4-bf307f734e80 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:31.024037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b942b-c0ae-4d4d-bcc4-bf307f734e80 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:31.024037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8b942b-c0ae-4d4d-bcc4-bf307f734e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:31.042005 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-7b8b942b-c0ae-4d4d-bcc4-bf307f734e80 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:48:31.043113 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-7b8b942b-c0ae-4d4d-bcc4-bf307f734e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:48:31.050902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 19:48:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:48:31.087546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745752c0-bcdd-4a7e-97de-4cf0a541c2ef None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:48:31.098398 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745752c0-bcdd-4a7e-97de-4cf0a541c2ef None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:48:31.102058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0a10b1-f75e-40af-a968-efa89e3c18f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:48:31.102721 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a10b1-f75e-40af-a968-efa89e3c18f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:31.103007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a10b1-f75e-40af-a968-efa89e3c18f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:31.103289 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0a10b1-f75e-40af-a968-efa89e3c18f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:31.112097 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a10b1-f75e-40af-a968-efa89e3c18f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:48:31.113563 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0a10b1-f75e-40af-a968-efa89e3c18f9 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:48:31.123316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 763/1533] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:48:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:48:34.852437 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7db4d-de57-4c29-a50c-0334f7d7c4c3 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:34.853004 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7db4d-de57-4c29-a50c-0334f7d7c4c3 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:34.853004 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7db4d-de57-4c29-a50c-0334f7d7c4c3 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:34.854311 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 19:48:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:48:34.859727 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228aa0c-ff48-4b53-9b63-15587acb6f9d None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:34.860171 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228aa0c-ff48-4b53-9b63-15587acb6f9d None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:34.860291 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228aa0c-ff48-4b53-9b63-15587acb6f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:34.878864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-4228aa0c-ff48-4b53-9b63-15587acb6f9d None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:48:34.880062 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-4228aa0c-ff48-4b53-9b63-15587acb6f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:48:34.888225 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 19:48:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:48:35.162233 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7db4d-de57-4c29-a50c-0334f7d7c4c3 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:48:35.164102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-c2b7db4d-de57-4c29-a50c-0334f7d7c4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:48:35.222868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7db4d-de57-4c29-a50c-0334f7d7c4c3 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:48:35.226519 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960d4fb-94b2-4665-bf0a-18e16ce9b6db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:48:35.227183 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960d4fb-94b2-4665-bf0a-18e16ce9b6db service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:35.227460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960d4fb-94b2-4665-bf0a-18e16ce9b6db service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:35.227735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960d4fb-94b2-4665-bf0a-18e16ce9b6db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:35.237262 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960d4fb-94b2-4665-bf0a-18e16ce9b6db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:48:35.238718 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960d4fb-94b2-4665-bf0a-18e16ce9b6db service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:48:35.244464 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:48:35.249812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 772/1536] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:48:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:48:37.495971 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d67d6-2aea-4c39-b2d9-ad966b1348ba None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:37.496510 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d67d6-2aea-4c39-b2d9-ad966b1348ba None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:37.496510 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d67d6-2aea-4c39-b2d9-ad966b1348ba None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:37.497925 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 19:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:48:37.503166 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a127ad-fcd5-4107-bfbb-7f90b282255f None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:37.503617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a127ad-fcd5-4107-bfbb-7f90b282255f None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:37.503723 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a127ad-fcd5-4107-bfbb-7f90b282255f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:37.521935 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-d2a127ad-fcd5-4107-bfbb-7f90b282255f None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:48:37.523622 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d2a127ad-fcd5-4107-bfbb-7f90b282255f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:48:37.531931 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 19:48:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:48:37.806913 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d67d6-2aea-4c39-b2d9-ad966b1348ba None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:48:37.808638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-706d67d6-2aea-4c39-b2d9-ad966b1348ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:48:37.822079 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d67d6-2aea-4c39-b2d9-ad966b1348ba None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:48:37.825601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c4d6d5-e19f-48ca-9935-32672adde6e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:48:37.826268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4d6d5-e19f-48ca-9935-32672adde6e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:48:37.826668 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4d6d5-e19f-48ca-9935-32672adde6e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:48:37.826950 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4d6d5-e19f-48ca-9935-32672adde6e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:48:37.836871 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4d6d5-e19f-48ca-9935-32672adde6e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:48:37.838351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c4d6d5-e19f-48ca-9935-32672adde6e6 service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:48:37.848457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 766/1539] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:48:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:48:47.516683 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:48:47.844307 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:10.020862 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09b04c-d0a6-4024-bda5-fa9740acdbd1 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.021550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09b04c-d0a6-4024-bda5-fa9740acdbd1 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.021550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd09b04c-d0a6-4024-bda5-fa9740acdbd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.053588 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-bd09b04c-d0a6-4024-bda5-fa9740acdbd1 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:10.054849 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-bd09b04c-d0a6-4024-bda5-fa9740acdbd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:10.068412 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 774/1540] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:10.078854 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d988b-d324-428c-89f6-93629e8e2f3a None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.079466 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d988b-d324-428c-89f6-93629e8e2f3a None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.079466 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0d988b-d324-428c-89f6-93629e8e2f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.103968 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-ad0d988b-d324-428c-89f6-93629e8e2f3a None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:10.105554 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-ad0d988b-d324-428c-89f6-93629e8e2f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:10.120280 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 767/1541] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:50:10.125369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09b04c-d0a6-4024-bda5-fa9740acdbd1 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:10.140643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd09b04c-d0a6-4024-bda5-fa9740acdbd1 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:10.149973 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3497a628-fd9b-463f-9d85-d62c413535fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:10.150818 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497a628-fd9b-463f-9d85-d62c413535fd admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.151109 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497a628-fd9b-463f-9d85-d62c413535fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.151391 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3497a628-fd9b-463f-9d85-d62c413535fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.152135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497a628-fd9b-463f-9d85-d62c413535fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:50:10.152448 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497a628-fd9b-463f-9d85-d62c413535fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:10.154116 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3497a628-fd9b-463f-9d85-d62c413535fd admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:10.174587 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 775/1542] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:50:10.183201 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d988b-d324-428c-89f6-93629e8e2f3a None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:10.197732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0d988b-d324-428c-89f6-93629e8e2f3a None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:10.205596 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7483321-88ab-4257-8865-7b56b6503753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:10.206575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7483321-88ab-4257-8865-7b56b6503753 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.206862 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7483321-88ab-4257-8865-7b56b6503753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.207140 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7483321-88ab-4257-8865-7b56b6503753 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.207856 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7483321-88ab-4257-8865-7b56b6503753 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:10.209859 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7483321-88ab-4257-8865-7b56b6503753 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:10.237958 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54900436-3542-4925-94f9-9fe3b883a1d6 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.238304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54900436-3542-4925-94f9-9fe3b883a1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.243236 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54900436-3542-4925-94f9-9fe3b883a1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.272781 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-54900436-3542-4925-94f9-9fe3b883a1d6 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:10.272781 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-54900436-3542-4925-94f9-9fe3b883a1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:10.286956 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 776/1543] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:50:10.301737 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7e4fb-c27e-4307-a136-d02d61acc8c8 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.305831 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7e4fb-c27e-4307-a136-d02d61acc8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.305831 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c7e4fb-c27e-4307-a136-d02d61acc8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.331017 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-95c7e4fb-c27e-4307-a136-d02d61acc8c8 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:10.332226 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-95c7e4fb-c27e-4307-a136-d02d61acc8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:10.340193 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 768/1544] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:10 2022] POST /identity/v3/projects => generated 375 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:10.341616 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b5206-a7aa-4abc-abad-495e87b17ca8 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.341917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b5206-a7aa-4abc-abad-495e87b17ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.342203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b5206-a7aa-4abc-abad-495e87b17ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.369920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-995b5206-a7aa-4abc-abad-495e87b17ca8 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:10.371146 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-995b5206-a7aa-4abc-abad-495e87b17ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:10.441455 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 769/1545] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:10.442291 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 777/1546] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:50:10.442984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13c6be-1a8d-4d11-b26e-8be5e44a1222 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.443285 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13c6be-1a8d-4d11-b26e-8be5e44a1222 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.443570 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13c6be-1a8d-4d11-b26e-8be5e44a1222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.444102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7e4fb-c27e-4307-a136-d02d61acc8c8 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:10.458606 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c7e4fb-c27e-4307-a136-d02d61acc8c8 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:10.465778 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622bed0a-c791-4a0d-9520-70899ae62cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:10.466471 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bed0a-c791-4a0d-9520-70899ae62cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.466756 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bed0a-c791-4a0d-9520-70899ae62cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.467034 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622bed0a-c791-4a0d-9520-70899ae62cc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.467728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622bed0a-c791-4a0d-9520-70899ae62cc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:10.469986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622bed0a-c791-4a0d-9520-70899ae62cc3 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:10.475760 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-fe13c6be-1a8d-4d11-b26e-8be5e44a1222 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:10.478034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-fe13c6be-1a8d-4d11-b26e-8be5e44a1222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:10.487766 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 770/1547] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:10.499376 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b266ba1-dd54-4270-88b7-814a8c455570 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.499376 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b266ba1-dd54-4270-88b7-814a8c455570 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.499376 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b266ba1-dd54-4270-88b7-814a8c455570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.509404 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-2b266ba1-dd54-4270-88b7-814a8c455570 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:10.511006 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-2b266ba1-dd54-4270-88b7-814a8c455570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:10.519087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 771/1548] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:10.520439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db1096-78f9-4dc3-9c7b-9072db9c91de None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.520738 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db1096-78f9-4dc3-9c7b-9072db9c91de None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.521242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db1096-78f9-4dc3-9c7b-9072db9c91de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.541326 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-d1db1096-78f9-4dc3-9c7b-9072db9c91de None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:10.542561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d1db1096-78f9-4dc3-9c7b-9072db9c91de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:10.550491 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 772/1549] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:10.551329 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db1096-78f9-4dc3-9c7b-9072db9c91de None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:10.560201 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 778/1550] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:10 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:10.560892 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db1096-78f9-4dc3-9c7b-9072db9c91de None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:10.561776 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafa42e-7174-47b5-97a0-ec6e9d99795e None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.562177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafa42e-7174-47b5-97a0-ec6e9d99795e None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.562464 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafa42e-7174-47b5-97a0-ec6e9d99795e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.565990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c08ce5-4061-4f9c-ade1-9b5597f4df42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:10.566651 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c08ce5-4061-4f9c-ade1-9b5597f4df42 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.566942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c08ce5-4061-4f9c-ade1-9b5597f4df42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.567223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c08ce5-4061-4f9c-ade1-9b5597f4df42 admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.567958 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c08ce5-4061-4f9c-ade1-9b5597f4df42 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:50:10.568297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c08ce5-4061-4f9c-ade1-9b5597f4df42 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:10.569349 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: WARNING py.warnings [None req-31c08ce5-4061-4f9c-ade1-9b5597f4df42 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 19:50:10.569349 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: warnings.warn(msg) May 27 19:50:10.569349 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: May 27 19:50:10.570252 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c08ce5-4061-4f9c-ade1-9b5597f4df42 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:10.580595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-ddafa42e-7174-47b5-97a0-ec6e9d99795e None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:10.587194 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ddafa42e-7174-47b5-97a0-ec6e9d99795e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:10.587354 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 773/1551] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:50:10.587354 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4249a10-2bac-46ff-a8fb-21fd8323e1b7 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.587354 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4249a10-2bac-46ff-a8fb-21fd8323e1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.587354 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4249a10-2bac-46ff-a8fb-21fd8323e1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.594088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 779/1552] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:10.598406 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776a5a6-e9c7-4650-b943-8088bb6c87a1 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.598691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776a5a6-e9c7-4650-b943-8088bb6c87a1 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.599017 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776a5a6-e9c7-4650-b943-8088bb6c87a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.606933 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-e4249a10-2bac-46ff-a8fb-21fd8323e1b7 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:10.608142 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-e4249a10-2bac-46ff-a8fb-21fd8323e1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:10.616296 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-0776a5a6-e9c7-4650-b943-8088bb6c87a1 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:10.616436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 774/1553] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:10.617486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-0776a5a6-e9c7-4650-b943-8088bb6c87a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:10.618128 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3c69d-a64c-48c7-b6ae-12d5afb1c0d5 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.618442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3c69d-a64c-48c7-b6ae-12d5afb1c0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.618723 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3c69d-a64c-48c7-b6ae-12d5afb1c0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.629932 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 780/1554] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:10.631296 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5cba4-5179-4e77-aaf2-734b5c047766 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.631584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5cba4-5179-4e77-aaf2-734b5c047766 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.631859 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5cba4-5179-4e77-aaf2-734b5c047766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.643235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-d5d3c69d-a64c-48c7-b6ae-12d5afb1c0d5 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:10.644428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d5d3c69d-a64c-48c7-b6ae-12d5afb1c0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:10.650621 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-52b5cba4-5179-4e77-aaf2-734b5c047766 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:10.651803 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-52b5cba4-5179-4e77-aaf2-734b5c047766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:10.652462 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 775/1555] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:10.653732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41934e-5e97-4da3-8724-fc865c7e5d32 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.654021 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41934e-5e97-4da3-8724-fc865c7e5d32 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.654300 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41934e-5e97-4da3-8724-fc865c7e5d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.659852 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 781/1556] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:10.660687 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5cba4-5179-4e77-aaf2-734b5c047766 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:10.673109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-3f41934e-5e97-4da3-8724-fc865c7e5d32 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:10.673313 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5cba4-5179-4e77-aaf2-734b5c047766 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:10.674298 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-3f41934e-5e97-4da3-8724-fc865c7e5d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:10.677735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c965a-c8ee-40f2-a47e-08767e421315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:10.678397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c965a-c8ee-40f2-a47e-08767e421315 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.678678 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c965a-c8ee-40f2-a47e-08767e421315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.678956 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13c965a-c8ee-40f2-a47e-08767e421315 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.679677 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13c965a-c8ee-40f2-a47e-08767e421315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:10.681959 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13c965a-c8ee-40f2-a47e-08767e421315 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:10.682235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 776/1557] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:10.683005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41934e-5e97-4da3-8724-fc865c7e5d32 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:10.693419 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 782/1558] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:10.693419 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c965a-c8ee-40f2-a47e-08767e421315 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:10.696620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41934e-5e97-4da3-8724-fc865c7e5d32 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:10.700190 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea91970-840c-4549-ab9b-0497fef6a706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:10.700877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea91970-840c-4549-ab9b-0497fef6a706 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.701200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea91970-840c-4549-ab9b-0497fef6a706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.701483 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea91970-840c-4549-ab9b-0497fef6a706 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.702305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea91970-840c-4549-ab9b-0497fef6a706 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:10.703633 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13c965a-c8ee-40f2-a47e-08767e421315 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:10.704233 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea91970-840c-4549-ab9b-0497fef6a706 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:10.708168 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ae5034-57d5-4105-af24-8ae30d8889dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:10.708807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae5034-57d5-4105-af24-8ae30d8889dc admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.709201 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae5034-57d5-4105-af24-8ae30d8889dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.709479 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ae5034-57d5-4105-af24-8ae30d8889dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.710198 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae5034-57d5-4105-af24-8ae30d8889dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:50:10.710510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae5034-57d5-4105-af24-8ae30d8889dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:10.712258 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ae5034-57d5-4105-af24-8ae30d8889dc admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:10.724620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 783/1559] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:50:10.725929 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d8fb2-e820-450a-97f1-cfb9e73f0c67 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.728711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d8fb2-e820-450a-97f1-cfb9e73f0c67 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.728711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d8fb2-e820-450a-97f1-cfb9e73f0c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.747954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-282d8fb2-e820-450a-97f1-cfb9e73f0c67 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:10.749208 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-282d8fb2-e820-450a-97f1-cfb9e73f0c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:10.756928 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 784/1560] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:10.758147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39db1fb-866d-4f13-a191-bbf4abf35ac4 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.758461 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39db1fb-866d-4f13-a191-bbf4abf35ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.758726 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39db1fb-866d-4f13-a191-bbf4abf35ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.775557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-b39db1fb-866d-4f13-a191-bbf4abf35ac4 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:10.776724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-b39db1fb-866d-4f13-a191-bbf4abf35ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:10.779052 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 777/1561] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:10 2022] POST /identity/v3/projects => generated 389 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:10.779883 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea91970-840c-4549-ab9b-0497fef6a706 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:10.837748 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:10.838726 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 785/1562] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:10.839257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39db1fb-866d-4f13-a191-bbf4abf35ac4 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:10.844907 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea91970-840c-4549-ab9b-0497fef6a706 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:10.848360 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39db1fb-866d-4f13-a191-bbf4abf35ac4 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:10.849717 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c57851-17d6-42b9-aa78-a46fe4144da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:10.852529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c57851-17d6-42b9-aa78-a46fe4144da4 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.852529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c57851-17d6-42b9-aa78-a46fe4144da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.852529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c57851-17d6-42b9-aa78-a46fe4144da4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.852529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c57851-17d6-42b9-aa78-a46fe4144da4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:50:10.852529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c57851-17d6-42b9-aa78-a46fe4144da4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:10.853930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f58edc-5f70-4bb3-b570-9221d619fb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:10.854534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c57851-17d6-42b9-aa78-a46fe4144da4 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:10.854646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f58edc-5f70-4bb3-b570-9221d619fb75 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.854857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f58edc-5f70-4bb3-b570-9221d619fb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.855135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f58edc-5f70-4bb3-b570-9221d619fb75 admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.855856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f58edc-5f70-4bb3-b570-9221d619fb75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:50:10.856165 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f58edc-5f70-4bb3-b570-9221d619fb75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:10.857997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f58edc-5f70-4bb3-b570-9221d619fb75 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:10.859650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:10.866457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 778/1563] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:50:10.870102 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c57851-17d6-42b9-aa78-a46fe4144da4 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:10.871511 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 786/1564] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:50:10.872278 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f58edc-5f70-4bb3-b570-9221d619fb75 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:10.880119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c57851-17d6-42b9-aa78-a46fe4144da4 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:10.882366 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f58edc-5f70-4bb3-b570-9221d619fb75 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:10.885117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0551b7-507a-4941-a4a0-d5fd4507b111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:10.885780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0551b7-507a-4941-a4a0-d5fd4507b111 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.886065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0551b7-507a-4941-a4a0-d5fd4507b111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.891692 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0551b7-507a-4941-a4a0-d5fd4507b111 admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.891692 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0551b7-507a-4941-a4a0-d5fd4507b111 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:50:10.891917 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b42e3-4ef8-4f00-b39f-4cca111bbf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:10.892036 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0551b7-507a-4941-a4a0-d5fd4507b111 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:10.892135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b42e3-4ef8-4f00-b39f-4cca111bbf66 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.892135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b42e3-4ef8-4f00-b39f-4cca111bbf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.892135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b42e3-4ef8-4f00-b39f-4cca111bbf66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.892427 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0551b7-507a-4941-a4a0-d5fd4507b111 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:10.892525 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b42e3-4ef8-4f00-b39f-4cca111bbf66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:10.892525 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b42e3-4ef8-4f00-b39f-4cca111bbf66 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:10.900569 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 779/1565] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:50:10.901444 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0551b7-507a-4941-a4a0-d5fd4507b111 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:10.902127 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 787/1566] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:10.903092 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b42e3-4ef8-4f00-b39f-4cca111bbf66 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:10.911262 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0551b7-507a-4941-a4a0-d5fd4507b111 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:10.913713 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b42e3-4ef8-4f00-b39f-4cca111bbf66 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:10.916682 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71674086-89d8-4ca6-8127-6f431e432d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:10.917373 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71674086-89d8-4ca6-8127-6f431e432d07 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.917658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71674086-89d8-4ca6-8127-6f431e432d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.917938 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71674086-89d8-4ca6-8127-6f431e432d07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.918667 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71674086-89d8-4ca6-8127-6f431e432d07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:10.920136 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc74ff5-f214-46cc-a919-073524252d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:10.920561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71674086-89d8-4ca6-8127-6f431e432d07 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:10.920940 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc74ff5-f214-46cc-a919-073524252d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.921316 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc74ff5-f214-46cc-a919-073524252d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.921671 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc74ff5-f214-46cc-a919-073524252d4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.922427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc74ff5-f214-46cc-a919-073524252d4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:50:10.922740 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc74ff5-f214-46cc-a919-073524252d4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:10.924901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc74ff5-f214-46cc-a919-073524252d4d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:10.936923 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 788/1567] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:50:10.937876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc74ff5-f214-46cc-a919-073524252d4d admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:10.947275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc74ff5-f214-46cc-a919-073524252d4d admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:10.951749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b8ba3-6392-4ae0-b2b8-2243da94b691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:10.952387 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b8ba3-6392-4ae0-b2b8-2243da94b691 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:10.952711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b8ba3-6392-4ae0-b2b8-2243da94b691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:10.953009 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b8ba3-6392-4ae0-b2b8-2243da94b691 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:10.953685 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b8ba3-6392-4ae0-b2b8-2243da94b691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:10.955574 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b8ba3-6392-4ae0-b2b8-2243da94b691 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:10.990311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 780/1568] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:10 2022] POST /identity/v3/projects => generated 385 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:10.991140 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71674086-89d8-4ca6-8127-6f431e432d07 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:11.053076 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71674086-89d8-4ca6-8127-6f431e432d07 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:11.057884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a379448-f713-40cd-b928-30ca2c66f309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:11.058545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a379448-f713-40cd-b928-30ca2c66f309 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:11.058834 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a379448-f713-40cd-b928-30ca2c66f309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:11.059116 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a379448-f713-40cd-b928-30ca2c66f309 admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:11.059921 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a379448-f713-40cd-b928-30ca2c66f309 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:50:11.060236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a379448-f713-40cd-b928-30ca2c66f309 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:11.061902 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a379448-f713-40cd-b928-30ca2c66f309 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:11.067832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 789/1569] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:10 2022] POST /identity/v3/users => generated 295 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:11.069030 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b8ba3-6392-4ae0-b2b8-2243da94b691 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:11.073834 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 781/1570] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:50:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:50:11.074897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a379448-f713-40cd-b928-30ca2c66f309 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:11.080066 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b8ba3-6392-4ae0-b2b8-2243da94b691 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:11.085599 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f68ea-bb1e-41d0-b2f5-d751dd37a4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:11.086188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a379448-f713-40cd-b928-30ca2c66f309 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:11.087143 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f68ea-bb1e-41d0-b2f5-d751dd37a4f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:11.087272 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f68ea-bb1e-41d0-b2f5-d751dd37a4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:11.087579 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f68ea-bb1e-41d0-b2f5-d751dd37a4f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:11.088327 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f68ea-bb1e-41d0-b2f5-d751dd37a4f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:11.089339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: WARNING py.warnings [None req-ce8f68ea-bb1e-41d0-b2f5-d751dd37a4f2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 19:50:11.089339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: warnings.warn(msg) May 27 19:50:11.089339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: May 27 19:50:11.090564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f68ea-bb1e-41d0-b2f5-d751dd37a4f2 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:11.091187 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced50c4-2660-49a5-a606-fe515937db52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:11.091923 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced50c4-2660-49a5-a606-fe515937db52 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:11.092206 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced50c4-2660-49a5-a606-fe515937db52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:11.092484 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced50c4-2660-49a5-a606-fe515937db52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:11.093583 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aced50c4-2660-49a5-a606-fe515937db52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:11.096516 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aced50c4-2660-49a5-a606-fe515937db52 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:11.190065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 790/1571] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:11 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:11.190954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f68ea-bb1e-41d0-b2f5-d751dd37a4f2 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:11.219478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 782/1572] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:11 2022] POST /identity/v3/projects => generated 365 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:11.220308 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced50c4-2660-49a5-a606-fe515937db52 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:11.256750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f68ea-bb1e-41d0-b2f5-d751dd37a4f2 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:11.282482 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced50c4-2660-49a5-a606-fe515937db52 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:11.287739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8005e-19c9-4404-b187-6bca29636359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:11.288642 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8005e-19c9-4404-b187-6bca29636359 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:11.288931 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8005e-19c9-4404-b187-6bca29636359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:11.289247 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8005e-19c9-4404-b187-6bca29636359 admin admin] PATH_INFO: `/v3/projects/f63fddb581024c34812bed88a9ad69d7/users/9845796304594bad9d36935e8e24448e/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:11.304534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d8005e-19c9-4404-b187-6bca29636359 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f63fddb581024c34812bed88a9ad69d7, user_id=9845796304594bad9d36935e8e24448e, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:11.307564 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d8005e-19c9-4404-b187-6bca29636359 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:11.315630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee94288-813f-4784-9a94-f6f9666b2497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:11.316354 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee94288-813f-4784-9a94-f6f9666b2497 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:11.316670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee94288-813f-4784-9a94-f6f9666b2497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:11.316916 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee94288-813f-4784-9a94-f6f9666b2497 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:11.317629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee94288-813f-4784-9a94-f6f9666b2497 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:11.319523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee94288-813f-4784-9a94-f6f9666b2497 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:11.377091 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 783/1573] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:11 2022] PUT /identity/v3/projects/f63fddb581024c34812bed88a9ad69d7/users/9845796304594bad9d36935e8e24448e/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:11.378156 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8005e-19c9-4404-b187-6bca29636359 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:11.391255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:11.399701 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 791/1574] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:11 2022] POST /identity/v3/projects => generated 373 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:11.399701 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee94288-813f-4784-9a94-f6f9666b2497 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:11.425472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:11.452683 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8005e-19c9-4404-b187-6bca29636359 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:11.463617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee94288-813f-4784-9a94-f6f9666b2497 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:11.469024 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189ee77-369b-46af-9851-5daaa8dd4fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:11.469708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189ee77-369b-46af-9851-5daaa8dd4fae admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:11.469993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189ee77-369b-46af-9851-5daaa8dd4fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:11.470552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6189ee77-369b-46af-9851-5daaa8dd4fae admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:11.470957 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189ee77-369b-46af-9851-5daaa8dd4fae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:11.472899 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6189ee77-369b-46af-9851-5daaa8dd4fae admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:11.511158 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc171ea-2f0a-4f8d-8726-20e4ccbee780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:11.511978 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc171ea-2f0a-4f8d-8726-20e4ccbee780 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:11.512094 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc171ea-2f0a-4f8d-8726-20e4ccbee780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:11.512356 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc171ea-2f0a-4f8d-8726-20e4ccbee780 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:11.513129 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc171ea-2f0a-4f8d-8726-20e4ccbee780 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:11.515065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc171ea-2f0a-4f8d-8726-20e4ccbee780 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:11.546322 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 792/1575] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:11 2022] POST /identity/v3/users => generated 293 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:11.546322 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189ee77-369b-46af-9851-5daaa8dd4fae admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:11.553651 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6189ee77-369b-46af-9851-5daaa8dd4fae admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:11.558469 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8956b-260a-4edf-bd56-9f23491ad29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:11.559145 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8956b-260a-4edf-bd56-9f23491ad29d admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:11.565136 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8956b-260a-4edf-bd56-9f23491ad29d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:11.565136 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8956b-260a-4edf-bd56-9f23491ad29d admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:11.565136 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8956b-260a-4edf-bd56-9f23491ad29d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:11.565136 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8956b-260a-4edf-bd56-9f23491ad29d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:11.572171 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 793/1576] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:11.572930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8956b-260a-4edf-bd56-9f23491ad29d admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:11.581955 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8956b-260a-4edf-bd56-9f23491ad29d admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:11.587074 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea78de7-5382-4b35-980c-809f1933c259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:11.588058 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea78de7-5382-4b35-980c-809f1933c259 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:11.588216 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea78de7-5382-4b35-980c-809f1933c259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:11.588409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea78de7-5382-4b35-980c-809f1933c259 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:11.589097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea78de7-5382-4b35-980c-809f1933c259 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:11.590940 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea78de7-5382-4b35-980c-809f1933c259 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:11.607547 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 784/1577] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:11 2022] POST /identity/v3/projects => generated 377 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:11.612552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc171ea-2f0a-4f8d-8726-20e4ccbee780 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:11.668735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc171ea-2f0a-4f8d-8726-20e4ccbee780 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:11.673532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7ad17-fbdc-4540-b2eb-3fd0882aabc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:11.674203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7ad17-fbdc-4540-b2eb-3fd0882aabc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:11.674491 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7ad17-fbdc-4540-b2eb-3fd0882aabc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:11.674780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7ad17-fbdc-4540-b2eb-3fd0882aabc8 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:11.675467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef7ad17-fbdc-4540-b2eb-3fd0882aabc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:11.676598 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: WARNING py.warnings [None req-6ef7ad17-fbdc-4540-b2eb-3fd0882aabc8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 19:50:11.676598 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: warnings.warn(msg) May 27 19:50:11.676598 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: May 27 19:50:11.677856 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef7ad17-fbdc-4540-b2eb-3fd0882aabc8 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:11.679625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 794/1578] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:11 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:11.680467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea78de7-5382-4b35-980c-809f1933c259 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:11.745521 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea78de7-5382-4b35-980c-809f1933c259 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:11.750043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f88b30-ce5f-4ccf-8440-0b8be45f4062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:11.750715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f88b30-ce5f-4ccf-8440-0b8be45f4062 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:11.751009 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f88b30-ce5f-4ccf-8440-0b8be45f4062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:11.751290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f88b30-ce5f-4ccf-8440-0b8be45f4062 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:11.751982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f88b30-ce5f-4ccf-8440-0b8be45f4062 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:11.754064 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f88b30-ce5f-4ccf-8440-0b8be45f4062 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:11.762766 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 785/1579] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:11 2022] POST /identity/v3/users => generated 287 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:11.764116 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b1176-eff6-4ab8-a098-4c976387b820 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:11.764935 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b1176-eff6-4ab8-a098-4c976387b820 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:11.765248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2b1176-eff6-4ab8-a098-4c976387b820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:11.793498 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-ca2b1176-eff6-4ab8-a098-4c976387b820 None None] MFA Rules not processed for user `9845796304594bad9d36935e8e24448e`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:11.794734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-ca2b1176-eff6-4ab8-a098-4c976387b820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:11.815503 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:11.860997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 795/1580] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:11 2022] POST /identity/v3/users => generated 283 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:11.860997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f88b30-ce5f-4ccf-8440-0b8be45f4062 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:11.866416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 786/1581] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:11.867310 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b1176-eff6-4ab8-a098-4c976387b820 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:11.870452 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f88b30-ce5f-4ccf-8440-0b8be45f4062 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:11.875140 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14509454-b76a-4a65-a4eb-21a3402b64fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:11.875813 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14509454-b76a-4a65-a4eb-21a3402b64fd admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:11.876101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14509454-b76a-4a65-a4eb-21a3402b64fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:11.876400 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14509454-b76a-4a65-a4eb-21a3402b64fd admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:11.877100 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14509454-b76a-4a65-a4eb-21a3402b64fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:11.879187 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14509454-b76a-4a65-a4eb-21a3402b64fd admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:11.884109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2b1176-eff6-4ab8-a098-4c976387b820 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:11.885959 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832cfbc-607f-4498-b5b5-f2e43d720a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:11.886624 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832cfbc-607f-4498-b5b5-f2e43d720a45 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:11.886907 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832cfbc-607f-4498-b5b5-f2e43d720a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:11.887211 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832cfbc-607f-4498-b5b5-f2e43d720a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:11.887939 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832cfbc-607f-4498-b5b5-f2e43d720a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:11.889599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7832cfbc-607f-4498-b5b5-f2e43d720a45 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:11.899630 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 787/1582] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:11.900417 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832cfbc-607f-4498-b5b5-f2e43d720a45 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:11.909586 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832cfbc-607f-4498-b5b5-f2e43d720a45 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:11.914071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e891f3-cc56-40db-bcb4-52cce312d914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:11.914741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e891f3-cc56-40db-bcb4-52cce312d914 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:11.915168 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e891f3-cc56-40db-bcb4-52cce312d914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:11.915450 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e891f3-cc56-40db-bcb4-52cce312d914 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:11.916153 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e891f3-cc56-40db-bcb4-52cce312d914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:11.917733 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e891f3-cc56-40db-bcb4-52cce312d914 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:11.928446 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 788/1583] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:11 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:11.929181 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e891f3-cc56-40db-bcb4-52cce312d914 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:11.939295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e891f3-cc56-40db-bcb4-52cce312d914 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:11.943801 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d85f0-a705-4458-a44d-f2acbc5cd2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:11.950343 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d85f0-a705-4458-a44d-f2acbc5cd2c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:11.950343 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d85f0-a705-4458-a44d-f2acbc5cd2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:11.950343 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d85f0-a705-4458-a44d-f2acbc5cd2c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:11.950343 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4d85f0-a705-4458-a44d-f2acbc5cd2c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:11.950343 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4d85f0-a705-4458-a44d-f2acbc5cd2c4 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:12.014208 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 796/1584] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:11 2022] POST /identity/v3/users => generated 287 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:12.014208 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14509454-b76a-4a65-a4eb-21a3402b64fd admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:12.021484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14509454-b76a-4a65-a4eb-21a3402b64fd admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:12.025987 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bb24d-3f8d-4c08-b0f1-543ce5633fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:12.026627 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bb24d-3f8d-4c08-b0f1-543ce5633fdc admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.027026 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bb24d-3f8d-4c08-b0f1-543ce5633fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.027488 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9bb24d-3f8d-4c08-b0f1-543ce5633fdc admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.028164 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9bb24d-3f8d-4c08-b0f1-543ce5633fdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:12.030130 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9bb24d-3f8d-4c08-b0f1-543ce5633fdc admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:12.063907 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 789/1585] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:11 2022] POST /identity/v3/users => generated 289 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:12.064913 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d85f0-a705-4458-a44d-f2acbc5cd2c4 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:12.074936 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d85f0-a705-4458-a44d-f2acbc5cd2c4 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:12.079407 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52630e4-b5ca-41f6-88fc-5b1c2d20cb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:12.080041 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52630e4-b5ca-41f6-88fc-5b1c2d20cb47 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.080324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52630e4-b5ca-41f6-88fc-5b1c2d20cb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.080600 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52630e4-b5ca-41f6-88fc-5b1c2d20cb47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.081365 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52630e4-b5ca-41f6-88fc-5b1c2d20cb47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:12.082959 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52630e4-b5ca-41f6-88fc-5b1c2d20cb47 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:12.094553 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 790/1586] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:12.095464 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52630e4-b5ca-41f6-88fc-5b1c2d20cb47 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:12.104790 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52630e4-b5ca-41f6-88fc-5b1c2d20cb47 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:12.109457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b17c4-d50b-49e2-ba9f-109f80a619ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:12.110085 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b17c4-d50b-49e2-ba9f-109f80a619ba admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.110366 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b17c4-d50b-49e2-ba9f-109f80a619ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.110643 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b17c4-d50b-49e2-ba9f-109f80a619ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.111347 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b17c4-d50b-49e2-ba9f-109f80a619ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:12.112896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b17c4-d50b-49e2-ba9f-109f80a619ba admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:12.114852 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 797/1587] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:12 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:12.115772 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bb24d-3f8d-4c08-b0f1-543ce5633fdc admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:12.126776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 791/1588] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:12.126776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b17c4-d50b-49e2-ba9f-109f80a619ba admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:12.127186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9bb24d-3f8d-4c08-b0f1-543ce5633fdc admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:12.131716 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447aaf09-f2a4-4ca9-9f32-06cb25d5ac3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:12.132356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447aaf09-f2a4-4ca9-9f32-06cb25d5ac3c admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.132640 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447aaf09-f2a4-4ca9-9f32-06cb25d5ac3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.132916 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447aaf09-f2a4-4ca9-9f32-06cb25d5ac3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.134280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447aaf09-f2a4-4ca9-9f32-06cb25d5ac3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:12.135528 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b17c4-d50b-49e2-ba9f-109f80a619ba admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:12.136242 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447aaf09-f2a4-4ca9-9f32-06cb25d5ac3c admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:12.140104 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86386780-874f-4426-8190-597724a53a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:12.140757 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86386780-874f-4426-8190-597724a53a93 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.141073 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86386780-874f-4426-8190-597724a53a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.141355 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86386780-874f-4426-8190-597724a53a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.142203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86386780-874f-4426-8190-597724a53a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:12.143804 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86386780-874f-4426-8190-597724a53a93 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:12.154182 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 792/1589] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:12.154964 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86386780-874f-4426-8190-597724a53a93 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:12.164259 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86386780-874f-4426-8190-597724a53a93 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:12.169562 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c3a3e-bf76-4cbc-b630-df3f9b112293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:12.170459 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c3a3e-bf76-4cbc-b630-df3f9b112293 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.170742 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c3a3e-bf76-4cbc-b630-df3f9b112293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.171020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c3a3e-bf76-4cbc-b630-df3f9b112293 admin admin] PATH_INFO: `/v3/projects/2efe5f8c717640528624dc103744dcf4/users/ed8754ddcfd54c24ac923db0fedc1606/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.187157 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c3a3e-bf76-4cbc-b630-df3f9b112293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2efe5f8c717640528624dc103744dcf4, user_id=ed8754ddcfd54c24ac923db0fedc1606, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:12.189769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c3a3e-bf76-4cbc-b630-df3f9b112293 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:12.207839 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 798/1590] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:12 2022] POST /identity/v3/projects => generated 377 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:12.208962 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447aaf09-f2a4-4ca9-9f32-06cb25d5ac3c admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:12.222311 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=86,keystone:INSERT=14 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:12.271968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447aaf09-f2a4-4ca9-9f32-06cb25d5ac3c admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:12.274458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 793/1591] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:12 2022] PUT /identity/v3/projects/2efe5f8c717640528624dc103744dcf4/users/ed8754ddcfd54c24ac923db0fedc1606/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:12.275383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c3a3e-bf76-4cbc-b630-df3f9b112293 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:12.331292 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27587934-4afb-426b-b7db-fd2f86b05922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:12.331968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27587934-4afb-426b-b7db-fd2f86b05922 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.332252 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27587934-4afb-426b-b7db-fd2f86b05922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.332528 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27587934-4afb-426b-b7db-fd2f86b05922 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.333285 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27587934-4afb-426b-b7db-fd2f86b05922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:12.334902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27587934-4afb-426b-b7db-fd2f86b05922 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:12.339653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c3a3e-bf76-4cbc-b630-df3f9b112293 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:12.345206 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e05d8-9c05-4a01-b265-0c5ba20ec8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:12.346083 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 799/1592] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:12.346508 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27587934-4afb-426b-b7db-fd2f86b05922 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:12.347738 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e05d8-9c05-4a01-b265-0c5ba20ec8cb admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.348168 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e05d8-9c05-4a01-b265-0c5ba20ec8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.348610 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e05d8-9c05-4a01-b265-0c5ba20ec8cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.349727 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e05d8-9c05-4a01-b265-0c5ba20ec8cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:12.351303 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e05d8-9c05-4a01-b265-0c5ba20ec8cb admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:12.357174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27587934-4afb-426b-b7db-fd2f86b05922 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:12.361816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818c19f-0c70-41ea-a4e4-d0e0cd1b936f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:12.362465 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 794/1593] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:12.362656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818c19f-0c70-41ea-a4e4-d0e0cd1b936f admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.362817 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818c19f-0c70-41ea-a4e4-d0e0cd1b936f admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.363119 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818c19f-0c70-41ea-a4e4-d0e0cd1b936f admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.363462 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e05d8-9c05-4a01-b265-0c5ba20ec8cb admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:12.363724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818c19f-0c70-41ea-a4e4-d0e0cd1b936f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:12.365516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818c19f-0c70-41ea-a4e4-d0e0cd1b936f admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:12.373934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e05d8-9c05-4a01-b265-0c5ba20ec8cb admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:12.379200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 800/1594] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:12.379200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818c19f-0c70-41ea-a4e4-d0e0cd1b936f admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:12.379436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46faf15-b370-4a02-8a86-93be0c9143c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:12.380122 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46faf15-b370-4a02-8a86-93be0c9143c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.380603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46faf15-b370-4a02-8a86-93be0c9143c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.380877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46faf15-b370-4a02-8a86-93be0c9143c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.381754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46faf15-b370-4a02-8a86-93be0c9143c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:12.383630 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46faf15-b370-4a02-8a86-93be0c9143c1 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:12.387178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818c19f-0c70-41ea-a4e4-d0e0cd1b936f admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:12.392185 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88e040-e536-4feb-8ed8-4a410f0c3377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:12.392841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88e040-e536-4feb-8ed8-4a410f0c3377 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.393161 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88e040-e536-4feb-8ed8-4a410f0c3377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.393442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88e040-e536-4feb-8ed8-4a410f0c3377 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.393960 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 795/1595] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:12.394169 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de88e040-e536-4feb-8ed8-4a410f0c3377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:12.394808 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46faf15-b370-4a02-8a86-93be0c9143c1 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:12.395776 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de88e040-e536-4feb-8ed8-4a410f0c3377 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:12.404538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46faf15-b370-4a02-8a86-93be0c9143c1 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:12.411313 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 801/1596] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:12.411313 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88e040-e536-4feb-8ed8-4a410f0c3377 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:12.411554 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a8949-65e0-4a6b-81cb-cfb93bb0ccc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:12.411554 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a8949-65e0-4a6b-81cb-cfb93bb0ccc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.411554 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a8949-65e0-4a6b-81cb-cfb93bb0ccc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.411554 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a8949-65e0-4a6b-81cb-cfb93bb0ccc5 admin admin] PATH_INFO: `/v3/projects/c994c57a63464397bd567b97dbc3097e/users/3c9aa6527c13466bbe5dce68a3b81ab5/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.420483 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88e040-e536-4feb-8ed8-4a410f0c3377 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:12.425317 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:12.425434 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae93bfe-2dd4-41b5-8b56-ca183a846ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:12.425835 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae93bfe-2dd4-41b5-8b56-ca183a846ccd admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.426119 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae93bfe-2dd4-41b5-8b56-ca183a846ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.426525 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae93bfe-2dd4-41b5-8b56-ca183a846ccd admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.427213 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae93bfe-2dd4-41b5-8b56-ca183a846ccd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:12.427365 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a8949-65e0-4a6b-81cb-cfb93bb0ccc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c994c57a63464397bd567b97dbc3097e, user_id=3c9aa6527c13466bbe5dce68a3b81ab5, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:12.429462 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae93bfe-2dd4-41b5-8b56-ca183a846ccd admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:12.430686 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95a8949-65e0-4a6b-81cb-cfb93bb0ccc5 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:12.502565 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 796/1597] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:12 2022] PUT /identity/v3/projects/c994c57a63464397bd567b97dbc3097e/users/3c9aa6527c13466bbe5dce68a3b81ab5/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:12.504140 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fcb15-9a55-450e-af55-b96169ec5fc6 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.504442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fcb15-9a55-450e-af55-b96169ec5fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.504724 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0fcb15-9a55-450e-af55-b96169ec5fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.522036 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-6f0fcb15-9a55-450e-af55-b96169ec5fc6 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:12.523236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-6f0fcb15-9a55-450e-af55-b96169ec5fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:12.536173 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 802/1598] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:12 2022] POST /identity/v3/users => generated 295 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:12.537046 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae93bfe-2dd4-41b5-8b56-ca183a846ccd admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:12.547573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae93bfe-2dd4-41b5-8b56-ca183a846ccd admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:12.583238 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 797/1599] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:12.584241 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fcb15-9a55-450e-af55-b96169ec5fc6 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:12.594733 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0fcb15-9a55-450e-af55-b96169ec5fc6 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:12.601546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f340192-373a-4c78-b5ee-612c7e37134f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:12.602869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f340192-373a-4c78-b5ee-612c7e37134f admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.603266 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f340192-373a-4c78-b5ee-612c7e37134f admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.611643 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f340192-373a-4c78-b5ee-612c7e37134f admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.611643 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f340192-373a-4c78-b5ee-612c7e37134f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:12.611643 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f340192-373a-4c78-b5ee-612c7e37134f admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:12.611950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6440757-4aff-48e4-9927-23730f4df07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:12.611950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6440757-4aff-48e4-9927-23730f4df07f admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.611950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6440757-4aff-48e4-9927-23730f4df07f admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.611950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6440757-4aff-48e4-9927-23730f4df07f admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.612340 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6440757-4aff-48e4-9927-23730f4df07f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:12.612340 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6440757-4aff-48e4-9927-23730f4df07f admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:12.616407 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 798/1600] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:12.617469 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f340192-373a-4c78-b5ee-612c7e37134f admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:12.621549 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 803/1601] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:12.622365 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6440757-4aff-48e4-9927-23730f4df07f admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:12.628010 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f340192-373a-4c78-b5ee-612c7e37134f admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:12.631796 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6440757-4aff-48e4-9927-23730f4df07f admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:12.633404 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a23c72-9327-4899-886a-cd30f3629518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:12.634322 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a23c72-9327-4899-886a-cd30f3629518 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.634604 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a23c72-9327-4899-886a-cd30f3629518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.634880 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a23c72-9327-4899-886a-cd30f3629518 admin admin] PATH_INFO: `/v3/projects/7241472ff5c44725bda3e741d79d474d/users/814653f467b14218849d4b7135da0f08/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.636365 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00aeed4-88a3-4b10-9857-2f1018636f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:12.637169 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00aeed4-88a3-4b10-9857-2f1018636f6c admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.637321 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00aeed4-88a3-4b10-9857-2f1018636f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.637596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00aeed4-88a3-4b10-9857-2f1018636f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.638319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00aeed4-88a3-4b10-9857-2f1018636f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:12.639946 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00aeed4-88a3-4b10-9857-2f1018636f6c admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:12.645086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:12.650211 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a23c72-9327-4899-886a-cd30f3629518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7241472ff5c44725bda3e741d79d474d, user_id=814653f467b14218849d4b7135da0f08, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:12.650876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 804/1602] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:12.651706 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00aeed4-88a3-4b10-9857-2f1018636f6c admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:12.652851 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a23c72-9327-4899-886a-cd30f3629518 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:12.661309 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00aeed4-88a3-4b10-9857-2f1018636f6c admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:12.666030 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b212eeb-cea5-4040-acf9-99510d54d79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:12.666905 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b212eeb-cea5-4040-acf9-99510d54d79e admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.667189 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b212eeb-cea5-4040-acf9-99510d54d79e admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.667466 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b212eeb-cea5-4040-acf9-99510d54d79e admin admin] PATH_INFO: `/v3/projects/0af218a6c1d24642b7948cde9d6fae28/users/acec6981edfe407286b27ad75d42eefc/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.683037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b212eeb-cea5-4040-acf9-99510d54d79e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0af218a6c1d24642b7948cde9d6fae28, user_id=acec6981edfe407286b27ad75d42eefc, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:12.684078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: WARNING py.warnings [None req-9b212eeb-cea5-4040-acf9-99510d54d79e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 19:50:12.684078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: warnings.warn(msg) May 27 19:50:12.684078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: May 27 19:50:12.686076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b212eeb-cea5-4040-acf9-99510d54d79e admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:12.753152 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 799/1603] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:12 2022] PUT /identity/v3/projects/7241472ff5c44725bda3e741d79d474d/users/814653f467b14218849d4b7135da0f08/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:12.754442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00515522-595c-4a15-9e96-e6bebcbcbe08 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.754737 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00515522-595c-4a15-9e96-e6bebcbcbe08 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.755034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00515522-595c-4a15-9e96-e6bebcbcbe08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.772517 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-00515522-595c-4a15-9e96-e6bebcbcbe08 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:12.773812 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-00515522-595c-4a15-9e96-e6bebcbcbe08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:12.776195 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 805/1604] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:12 2022] PUT /identity/v3/projects/0af218a6c1d24642b7948cde9d6fae28/users/acec6981edfe407286b27ad75d42eefc/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:12.777084 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b212eeb-cea5-4040-acf9-99510d54d79e admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:12.838356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b212eeb-cea5-4040-acf9-99510d54d79e admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:12.842922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14bb11-1df3-468b-9145-d80fc8f58474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:12.843688 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 800/1605] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:12.843832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14bb11-1df3-468b-9145-d80fc8f58474 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.844148 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14bb11-1df3-468b-9145-d80fc8f58474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.844433 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14bb11-1df3-468b-9145-d80fc8f58474 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.844531 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00515522-595c-4a15-9e96-e6bebcbcbe08 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:12.845308 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff14bb11-1df3-468b-9145-d80fc8f58474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:12.846972 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff14bb11-1df3-468b-9145-d80fc8f58474 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:12.858097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 806/1606] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:12.858774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14bb11-1df3-468b-9145-d80fc8f58474 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:12.868851 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14bb11-1df3-468b-9145-d80fc8f58474 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:12.873632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4a9b9-c827-460c-b3af-b9c708617ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:12.874568 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4a9b9-c827-460c-b3af-b9c708617ce6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.874930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4a9b9-c827-460c-b3af-b9c708617ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.875213 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4a9b9-c827-460c-b3af-b9c708617ce6 admin admin] PATH_INFO: `/v3/projects/c4362059e243479193d5a02e2e5611a1/users/10369664f1914c51893eef1929b7d26b/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.890525 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c4a9b9-c827-460c-b3af-b9c708617ce6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4362059e243479193d5a02e2e5611a1, user_id=10369664f1914c51893eef1929b7d26b, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:12.893242 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c4a9b9-c827-460c-b3af-b9c708617ce6 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:12.901360 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00515522-595c-4a15-9e96-e6bebcbcbe08 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:12.905384 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601efeb4-dcf1-421f-90e7-60b9ea5fa4b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:12.906073 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601efeb4-dcf1-421f-90e7-60b9ea5fa4b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.906359 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601efeb4-dcf1-421f-90e7-60b9ea5fa4b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.906638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601efeb4-dcf1-421f-90e7-60b9ea5fa4b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.913772 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:12.916596 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601efeb4-dcf1-421f-90e7-60b9ea5fa4b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:12.918058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601efeb4-dcf1-421f-90e7-60b9ea5fa4b7 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:12.927502 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 801/1607] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:12.932604 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601efeb4-dcf1-421f-90e7-60b9ea5fa4b7 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:12.937869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601efeb4-dcf1-421f-90e7-60b9ea5fa4b7 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:12.942686 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb401f18-3f3f-4422-b18c-2d5823b6c631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:12.943579 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb401f18-3f3f-4422-b18c-2d5823b6c631 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:12.943869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb401f18-3f3f-4422-b18c-2d5823b6c631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:12.944152 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb401f18-3f3f-4422-b18c-2d5823b6c631 admin admin] PATH_INFO: `/v3/projects/340927ea7f5741ea8da66fa6fc237a32/users/5b66b3e0d7ea43079df33888f68b9fd6/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:12.959815 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 807/1608] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:12 2022] PUT /identity/v3/projects/c4362059e243479193d5a02e2e5611a1/users/10369664f1914c51893eef1929b7d26b/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:12.964326 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4a9b9-c827-460c-b3af-b9c708617ce6 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:12.965990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb401f18-3f3f-4422-b18c-2d5823b6c631 admin admin] RBAC: Authorizing `identity:create_grant(project_id=340927ea7f5741ea8da66fa6fc237a32, user_id=5b66b3e0d7ea43079df33888f68b9fd6, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:12.969902 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb401f18-3f3f-4422-b18c-2d5823b6c631 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:13.032010 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4a9b9-c827-460c-b3af-b9c708617ce6 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:13.036629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f96057-3bbd-421b-97a5-e8db05b12d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:13.037550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f96057-3bbd-421b-97a5-e8db05b12d1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:13.037839 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f96057-3bbd-421b-97a5-e8db05b12d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:13.038220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f96057-3bbd-421b-97a5-e8db05b12d1c admin admin] PATH_INFO: `/v3/projects/23e6266a43c845acaae1c2c2a1ffbfe8/users/a3ef31e3eac944898442f7a273334ec0/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:13.063220 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 802/1609] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:12 2022] PUT /identity/v3/projects/340927ea7f5741ea8da66fa6fc237a32/users/5b66b3e0d7ea43079df33888f68b9fd6/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:13.063220 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81fc13-5c8c-4da5-9337-2c6a4f4ea061 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:13.063550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f96057-3bbd-421b-97a5-e8db05b12d1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=23e6266a43c845acaae1c2c2a1ffbfe8, user_id=a3ef31e3eac944898442f7a273334ec0, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:13.063651 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81fc13-5c8c-4da5-9337-2c6a4f4ea061 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:13.063651 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81fc13-5c8c-4da5-9337-2c6a4f4ea061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:13.063951 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f96057-3bbd-421b-97a5-e8db05b12d1c admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:13.089861 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-da81fc13-5c8c-4da5-9337-2c6a4f4ea061 None None] MFA Rules not processed for user `814653f467b14218849d4b7135da0f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:13.091102 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-da81fc13-5c8c-4da5-9337-2c6a4f4ea061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:13.143929 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 808/1610] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:12 2022] PUT /identity/v3/projects/23e6266a43c845acaae1c2c2a1ffbfe8/users/a3ef31e3eac944898442f7a273334ec0/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:13.146384 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022cd04-0d8f-4373-9532-6235063510d9 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:13.146738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022cd04-0d8f-4373-9532-6235063510d9 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:13.147125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0022cd04-0d8f-4373-9532-6235063510d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:13.167378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 803/1611] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:13.168198 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81fc13-5c8c-4da5-9337-2c6a4f4ea061 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:13.182702 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-0022cd04-0d8f-4373-9532-6235063510d9 None None] MFA Rules not processed for user `acec6981edfe407286b27ad75d42eefc`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:13.183902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-0022cd04-0d8f-4373-9532-6235063510d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:13.203184 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:13.226832 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81fc13-5c8c-4da5-9337-2c6a4f4ea061 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:13.256399 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 809/1612] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:13.257796 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022cd04-0d8f-4373-9532-6235063510d9 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:13.306510 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc30b5-11c2-418b-887a-58d471e841aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:13.306510 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc30b5-11c2-418b-887a-58d471e841aa service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:13.306510 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc30b5-11c2-418b-887a-58d471e841aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:13.306510 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc30b5-11c2-418b-887a-58d471e841aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:13.340521 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0022cd04-0d8f-4373-9532-6235063510d9 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:13.344861 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:13.345736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f2388-2186-4d91-adb6-eefdab244ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:13.346635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f2388-2186-4d91-adb6-eefdab244ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:13.346929 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f2388-2186-4d91-adb6-eefdab244ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:13.347204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f2388-2186-4d91-adb6-eefdab244ca2 admin admin] PATH_INFO: `/v3/projects/2ad70bc1d6d64d1ea0a864d5b3ef231c/users/3cff73dd9c084ea29792bb83f012edeb/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:13.362245 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f2388-2186-4d91-adb6-eefdab244ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ad70bc1d6d64d1ea0a864d5b3ef231c, user_id=3cff73dd9c084ea29792bb83f012edeb, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:13.364968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f2388-2186-4d91-adb6-eefdab244ca2 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:13.380998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc30b5-11c2-418b-887a-58d471e841aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:13.382007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc30b5-11c2-418b-887a-58d471e841aa service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:13.392837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 804/1613] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:13.394147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0ce37-b375-4dbe-b7a8-db0d11d149ed None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:13.394445 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0ce37-b375-4dbe-b7a8-db0d11d149ed None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:13.394730 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d0ce37-b375-4dbe-b7a8-db0d11d149ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:13.411803 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-e9d0ce37-b375-4dbe-b7a8-db0d11d149ed None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:13.413089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-e9d0ce37-b375-4dbe-b7a8-db0d11d149ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:13.421440 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 805/1614] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:13.422692 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26162c-8d4f-4da7-9a75-b0a6cd128d08 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:13.422823 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26162c-8d4f-4da7-9a75-b0a6cd128d08 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:13.423111 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26162c-8d4f-4da7-9a75-b0a6cd128d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:13.451270 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-9a26162c-8d4f-4da7-9a75-b0a6cd128d08 None None] MFA Rules not processed for user `5b66b3e0d7ea43079df33888f68b9fd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:13.452480 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-9a26162c-8d4f-4da7-9a75-b0a6cd128d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:13.461414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 810/1615] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:13 2022] PUT /identity/v3/projects/2ad70bc1d6d64d1ea0a864d5b3ef231c/users/3cff73dd9c084ea29792bb83f012edeb/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:13.465701 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d8231-af13-4177-b85f-5bd92b95c42c None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:13.466065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d8231-af13-4177-b85f-5bd92b95c42c None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:13.466419 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675d8231-af13-4177-b85f-5bd92b95c42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:13.493249 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-675d8231-af13-4177-b85f-5bd92b95c42c None None] MFA Rules not processed for user `a3ef31e3eac944898442f7a273334ec0`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:13.495148 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-675d8231-af13-4177-b85f-5bd92b95c42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:13.518247 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 806/1616] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:13.522161 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991b55d-040e-4e22-93f2-bc038de6907c None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:13.522161 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991b55d-040e-4e22-93f2-bc038de6907c None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:13.522161 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991b55d-040e-4e22-93f2-bc038de6907c None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:13.522562 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 807/1617] 10.222.0.13 () {54 vars in 961 bytes} [Fri May 27 19:50:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:50:13.523262 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991b55d-040e-4e22-93f2-bc038de6907c None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:13.569829 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 811/1618] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:13.572039 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d8231-af13-4177-b85f-5bd92b95c42c None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:13.573937 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-675d8231-af13-4177-b85f-5bd92b95c42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:13.604882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991b55d-040e-4e22-93f2-bc038de6907c None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:13.611807 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78968581-7b83-4cff-a664-101410fae8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:13.613092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78968581-7b83-4cff-a664-101410fae8d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:13.613092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78968581-7b83-4cff-a664-101410fae8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:13.613517 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78968581-7b83-4cff-a664-101410fae8d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:13.614741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78968581-7b83-4cff-a664-101410fae8d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:13.619206 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78968581-7b83-4cff-a664-101410fae8d2 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:13.621056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:13.646623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675d8231-af13-4177-b85f-5bd92b95c42c None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:13.650463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43287d2c-37f9-4cc0-9e7e-79f66e106ca2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:13.651151 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43287d2c-37f9-4cc0-9e7e-79f66e106ca2 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:13.651428 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43287d2c-37f9-4cc0-9e7e-79f66e106ca2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:13.652257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43287d2c-37f9-4cc0-9e7e-79f66e106ca2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:13.663677 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43287d2c-37f9-4cc0-9e7e-79f66e106ca2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:13.666333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43287d2c-37f9-4cc0-9e7e-79f66e106ca2 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:13.677297 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 812/1619] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:13.678154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43287d2c-37f9-4cc0-9e7e-79f66e106ca2 service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:13.688705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43287d2c-37f9-4cc0-9e7e-79f66e106ca2 service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:13.694034 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7c0f1-a10d-45df-bef9-6d1ca9d9c253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:13.708120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7c0f1-a10d-45df-bef9-6d1ca9d9c253 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:13.708120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7c0f1-a10d-45df-bef9-6d1ca9d9c253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:13.708120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd7c0f1-a10d-45df-bef9-6d1ca9d9c253 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:13.708120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7c0f1-a10d-45df-bef9-6d1ca9d9c253 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:13.708120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd7c0f1-a10d-45df-bef9-6d1ca9d9c253 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:13.709396 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 813/1620] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:13.710216 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7c0f1-a10d-45df-bef9-6d1ca9d9c253 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:13.720084 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd7c0f1-a10d-45df-bef9-6d1ca9d9c253 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:13.724881 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826ec65-3ce5-4649-b6ba-ced4c06e7f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:13.726592 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826ec65-3ce5-4649-b6ba-ced4c06e7f74 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:13.726592 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826ec65-3ce5-4649-b6ba-ced4c06e7f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:13.726592 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826ec65-3ce5-4649-b6ba-ced4c06e7f74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:13.727784 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826ec65-3ce5-4649-b6ba-ced4c06e7f74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:13.732432 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 808/1621] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:13 2022] POST /identity/v3/projects => generated 363 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:13.732612 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826ec65-3ce5-4649-b6ba-ced4c06e7f74 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:13.732731 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f24ef0-7691-4948-8df0-137f719aa3aa None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:13.732731 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f24ef0-7691-4948-8df0-137f719aa3aa None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:13.732731 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f24ef0-7691-4948-8df0-137f719aa3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:13.751037 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-48f24ef0-7691-4948-8df0-137f719aa3aa None None] MFA Rules not processed for user `7542e1fd160e4b2c9d25accf19aad19f`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:13.753195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-48f24ef0-7691-4948-8df0-137f719aa3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:13.826928 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 814/1622] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:13 2022] POST /identity/v3/projects => generated 369 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:13.828322 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d79615-d874-48a3-ab67-b19eec742f8e None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:13.828613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d79615-d874-48a3-ab67-b19eec742f8e None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:13.828899 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d79615-d874-48a3-ab67-b19eec742f8e None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:13.833329 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 815/1623] 10.222.0.13 () {54 vars in 961 bytes} [Fri May 27 19:50:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:50:13.834180 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d79615-d874-48a3-ab67-b19eec742f8e None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:13.856627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 809/1624] 10.222.0.13 () {58 vars in 1058 bytes} [Fri May 27 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:13.869380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f24ef0-7691-4948-8df0-137f719aa3aa None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:13.930380 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d79615-d874-48a3-ab67-b19eec742f8e None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:13.942198 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec1b03a-c233-41e0-ba70-2740d14ee6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:13.943563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1b03a-c233-41e0-ba70-2740d14ee6e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:13.944206 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1b03a-c233-41e0-ba70-2740d14ee6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:13.944722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1b03a-c233-41e0-ba70-2740d14ee6e2 admin admin] PATH_INFO: `/v3/projects/2ad70bc1d6d64d1ea0a864d5b3ef231c/users/3cff73dd9c084ea29792bb83f012edeb/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:13.948675 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec1b03a-c233-41e0-ba70-2740d14ee6e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ad70bc1d6d64d1ea0a864d5b3ef231c, user_id=3cff73dd9c084ea29792bb83f012edeb, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:13.952004 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec1b03a-c233-41e0-ba70-2740d14ee6e2 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:13.957476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f24ef0-7691-4948-8df0-137f719aa3aa None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:13.962957 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0501c14c-7123-4a4e-a98b-94e28e8368d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:13.963659 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501c14c-7123-4a4e-a98b-94e28e8368d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:13.963928 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501c14c-7123-4a4e-a98b-94e28e8368d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:13.964258 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0501c14c-7123-4a4e-a98b-94e28e8368d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:13.964969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0501c14c-7123-4a4e-a98b-94e28e8368d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:13.966851 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 816/1625] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:13 2022] PUT /identity/v3/projects/2ad70bc1d6d64d1ea0a864d5b3ef231c/users/3cff73dd9c084ea29792bb83f012edeb/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:13.967555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec1b03a-c233-41e0-ba70-2740d14ee6e2 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:13.969321 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0501c14c-7123-4a4e-a98b-94e28e8368d3 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:14.043422 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec1b03a-c233-41e0-ba70-2740d14ee6e2 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:14.048972 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89124a71-003a-475d-bbbe-d13ec8fe841d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:14.049665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89124a71-003a-475d-bbbe-d13ec8fe841d admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:14.049956 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89124a71-003a-475d-bbbe-d13ec8fe841d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:14.050239 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89124a71-003a-475d-bbbe-d13ec8fe841d admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:14.050943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89124a71-003a-475d-bbbe-d13ec8fe841d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:14.059826 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89124a71-003a-475d-bbbe-d13ec8fe841d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:14.060031 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 810/1626] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:13 2022] POST /identity/v3/users => generated 288 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:14.060031 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d51ef-a9f1-4136-8a4f-28659d612fa0 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:14.060031 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d51ef-a9f1-4136-8a4f-28659d612fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:14.060031 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88d51ef-a9f1-4136-8a4f-28659d612fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:14.078557 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-e88d51ef-a9f1-4136-8a4f-28659d612fa0 None None] MFA Rules not processed for user `7542e1fd160e4b2c9d25accf19aad19f`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:14.079753 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-e88d51ef-a9f1-4136-8a4f-28659d612fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:14.088471 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:14.170752 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 811/1627] 10.222.0.13 () {58 vars in 1058 bytes} [Fri May 27 19:50:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:14.171834 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d51ef-a9f1-4136-8a4f-28659d612fa0 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:14.198963 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 817/1628] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:13 2022] POST /identity/v3/users => generated 291 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:14.200298 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88d51ef-a9f1-4136-8a4f-28659d612fa0 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:14.200383 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee24f95-1238-4c6d-8447-d4457d4060f6 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:14.200383 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee24f95-1238-4c6d-8447-d4457d4060f6 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:14.200383 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee24f95-1238-4c6d-8447-d4457d4060f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:14.200746 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df79bc-7b63-4e3e-92f9-3dedd754b4f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:14.200746 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df79bc-7b63-4e3e-92f9-3dedd754b4f4 service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:14.200746 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df79bc-7b63-4e3e-92f9-3dedd754b4f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:14.200746 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4df79bc-7b63-4e3e-92f9-3dedd754b4f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:14.225839 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-2ee24f95-1238-4c6d-8447-d4457d4060f6 None None] MFA Rules not processed for user `3cff73dd9c084ea29792bb83f012edeb`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:14.227375 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-2ee24f95-1238-4c6d-8447-d4457d4060f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:14.273459 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df79bc-7b63-4e3e-92f9-3dedd754b4f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:14.275215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4df79bc-7b63-4e3e-92f9-3dedd754b4f4 service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:14.287678 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 812/1629] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:14.289932 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:14.296103 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df79bc-7b63-4e3e-92f9-3dedd754b4f4 service nova] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:14.315623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4df79bc-7b63-4e3e-92f9-3dedd754b4f4 service nova] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:14.326206 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7507b-017d-410c-9972-434778aeda17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:14.331575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7507b-017d-410c-9972-434778aeda17 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:14.331575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7507b-017d-410c-9972-434778aeda17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:14.331575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f7507b-017d-410c-9972-434778aeda17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:14.331575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f7507b-017d-410c-9972-434778aeda17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:14.331575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f7507b-017d-410c-9972-434778aeda17 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:14.339723 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 818/1630] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:14 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:14.340585 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee24f95-1238-4c6d-8447-d4457d4060f6 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:14.342510 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 813/1631] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:14.348015 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7507b-017d-410c-9972-434778aeda17 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:14.359040 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee24f95-1238-4c6d-8447-d4457d4060f6 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:14.369236 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40014e9c-5796-4832-a073-5bd163d5dc23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:14.370595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40014e9c-5796-4832-a073-5bd163d5dc23 service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:14.371042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40014e9c-5796-4832-a073-5bd163d5dc23 service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:14.371885 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40014e9c-5796-4832-a073-5bd163d5dc23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:14.373934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f7507b-017d-410c-9972-434778aeda17 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:14.379423 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0ecd4-25d1-454f-8285-244320bb8cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:14.380095 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0ecd4-25d1-454f-8285-244320bb8cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:14.380370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0ecd4-25d1-454f-8285-244320bb8cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:14.380657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0ecd4-25d1-454f-8285-244320bb8cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:14.381408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e0ecd4-25d1-454f-8285-244320bb8cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:14.383030 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e0ecd4-25d1-454f-8285-244320bb8cc9 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:14.399493 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 814/1632] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:14.399493 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0ecd4-25d1-454f-8285-244320bb8cc9 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:14.412057 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0ecd4-25d1-454f-8285-244320bb8cc9 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:14.418211 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dca755-2316-44fa-9104-7a6a7a796de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:14.419110 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dca755-2316-44fa-9104-7a6a7a796de2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:14.419393 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dca755-2316-44fa-9104-7a6a7a796de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:14.419721 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dca755-2316-44fa-9104-7a6a7a796de2 admin admin] PATH_INFO: `/v3/projects/b0106f670f1240ad96c504fed8bb5fb8/users/eb9a5b210d504772b120e868c42d9ece/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:14.437614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dca755-2316-44fa-9104-7a6a7a796de2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0106f670f1240ad96c504fed8bb5fb8, user_id=eb9a5b210d504772b120e868c42d9ece, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:14.440806 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dca755-2316-44fa-9104-7a6a7a796de2 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:14.448189 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40014e9c-5796-4832-a073-5bd163d5dc23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:14.450459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40014e9c-5796-4832-a073-5bd163d5dc23 service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:14.462384 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 819/1633] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:14.463382 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40014e9c-5796-4832-a073-5bd163d5dc23 service nova] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:14.521100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 815/1634] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:14 2022] PUT /identity/v3/projects/b0106f670f1240ad96c504fed8bb5fb8/users/eb9a5b210d504772b120e868c42d9ece/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:14.521100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dca755-2316-44fa-9104-7a6a7a796de2 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:14.550280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40014e9c-5796-4832-a073-5bd163d5dc23 service nova] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:14.634829 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f3d65-78ab-4988-9d3c-6f0440e4a6b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:14.635517 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f3d65-78ab-4988-9d3c-6f0440e4a6b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:14.635809 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f3d65-78ab-4988-9d3c-6f0440e4a6b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:14.636092 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f3d65-78ab-4988-9d3c-6f0440e4a6b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:14.703378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dca755-2316-44fa-9104-7a6a7a796de2 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:14.730491 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bdff54-6928-487a-bef5-82a637a2fbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:14.731176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdff54-6928-487a-bef5-82a637a2fbe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:14.731345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdff54-6928-487a-bef5-82a637a2fbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:14.731769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdff54-6928-487a-bef5-82a637a2fbe6 admin admin] PATH_INFO: `/v3/projects/2fe434ee7dc74190b66e1875bfa986e8/users/d73c96ae7b444d8287b96b0f24f81d47/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:14.760277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:14.768486 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bdff54-6928-487a-bef5-82a637a2fbe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fe434ee7dc74190b66e1875bfa986e8, user_id=d73c96ae7b444d8287b96b0f24f81d47, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:14.771715 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bdff54-6928-487a-bef5-82a637a2fbe6 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:14.819657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f3d65-78ab-4988-9d3c-6f0440e4a6b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:14.822293 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f3d65-78ab-4988-9d3c-6f0440e4a6b4 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:14.830706 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:14.839073 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 820/1635] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:14.840051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f3d65-78ab-4988-9d3c-6f0440e4a6b4 service cinder] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:14.858227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f3d65-78ab-4988-9d3c-6f0440e4a6b4 service cinder] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:14.864461 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28cad1e-6d23-44ab-8134-99aead6c1b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:14.865685 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cad1e-6d23-44ab-8134-99aead6c1b15 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:14.866166 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cad1e-6d23-44ab-8134-99aead6c1b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:14.866597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cad1e-6d23-44ab-8134-99aead6c1b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:14.868068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28cad1e-6d23-44ab-8134-99aead6c1b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:14.870248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28cad1e-6d23-44ab-8134-99aead6c1b15 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:14.884218 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 821/1636] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:14.889178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835a09b-bfee-462f-b663-d7dc0e586159 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:14.889178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835a09b-bfee-462f-b663-d7dc0e586159 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:14.889178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835a09b-bfee-462f-b663-d7dc0e586159 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:14.889178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 822/1637] 10.222.0.13 () {54 vars in 912 bytes} [Fri May 27 19:50:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:50:14.890923 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b17c27-c43c-42bd-b38b-5b935eaf5d40 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:14.891681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b17c27-c43c-42bd-b38b-5b935eaf5d40 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:14.892124 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b17c27-c43c-42bd-b38b-5b935eaf5d40 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:14.900888 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 823/1638] 10.222.0.13 () {54 vars in 912 bytes} [Fri May 27 19:50:14 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:50:14.902031 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b17c27-c43c-42bd-b38b-5b935eaf5d40 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:14.913093 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 816/1639] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:14 2022] PUT /identity/v3/projects/2fe434ee7dc74190b66e1875bfa986e8/users/d73c96ae7b444d8287b96b0f24f81d47/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:14.914565 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316d0cc-ab3f-4da0-8ea3-c69a5ed3642f None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:14.917082 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316d0cc-ab3f-4da0-8ea3-c69a5ed3642f None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:14.917082 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316d0cc-ab3f-4da0-8ea3-c69a5ed3642f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:14.923737 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b17c27-c43c-42bd-b38b-5b935eaf5d40 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:14.962748 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-7316d0cc-ab3f-4da0-8ea3-c69a5ed3642f None None] MFA Rules not processed for user `7542e1fd160e4b2c9d25accf19aad19f`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:14.964222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-7316d0cc-ab3f-4da0-8ea3-c69a5ed3642f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:15.028726 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d74198-6f00-4aed-acf1-3d42af866074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:15.031017 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d74198-6f00-4aed-acf1-3d42af866074 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:15.032148 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d74198-6f00-4aed-acf1-3d42af866074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:15.032755 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d74198-6f00-4aed-acf1-3d42af866074 admin admin] PATH_INFO: `/v3/projects/b0106f670f1240ad96c504fed8bb5fb8/users/eb9a5b210d504772b120e868c42d9ece/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:15.038558 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d74198-6f00-4aed-acf1-3d42af866074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0106f670f1240ad96c504fed8bb5fb8, user_id=eb9a5b210d504772b120e868c42d9ece, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:15.046492 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d74198-6f00-4aed-acf1-3d42af866074 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:15.075218 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 824/1640] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:14 2022] PUT /identity/v3/projects/b0106f670f1240ad96c504fed8bb5fb8/users/eb9a5b210d504772b120e868c42d9ece/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:15.076261 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c51f2-c7b9-453d-90fe-313c511b8840 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:15.076614 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c51f2-c7b9-453d-90fe-313c511b8840 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:15.077519 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c51f2-c7b9-453d-90fe-313c511b8840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:15.100387 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 817/1641] 10.222.0.13 () {58 vars in 1009 bytes} [Fri May 27 19:50:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:15.101385 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316d0cc-ab3f-4da0-8ea3-c69a5ed3642f None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:15.116502 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316d0cc-ab3f-4da0-8ea3-c69a5ed3642f None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:15.122494 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-4e6c51f2-c7b9-453d-90fe-313c511b8840 None None] MFA Rules not processed for user `7542e1fd160e4b2c9d25accf19aad19f`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:15.123711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-4e6c51f2-c7b9-453d-90fe-313c511b8840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:15.193130 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3c823-ba78-4f7a-a9ad-986f86d9b79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:15.193130 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3c823-ba78-4f7a-a9ad-986f86d9b79b admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:15.193130 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3c823-ba78-4f7a-a9ad-986f86d9b79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:15.193130 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3c823-ba78-4f7a-a9ad-986f86d9b79b admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:15.193937 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3c823-ba78-4f7a-a9ad-986f86d9b79b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:15.195387 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3c823-ba78-4f7a-a9ad-986f86d9b79b admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:15.214178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 818/1642] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:15.215926 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110758d-8d97-467a-9a5c-ce8c8d4bf1ef None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:15.216172 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110758d-8d97-467a-9a5c-ce8c8d4bf1ef None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:15.216456 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110758d-8d97-467a-9a5c-ce8c8d4bf1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:15.246028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-d110758d-8d97-467a-9a5c-ce8c8d4bf1ef None None] MFA Rules not processed for user `eb9a5b210d504772b120e868c42d9ece`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:15.247255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d110758d-8d97-467a-9a5c-ce8c8d4bf1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:15.290408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 825/1643] 10.222.0.13 () {58 vars in 1009 bytes} [Fri May 27 19:50:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:15.291944 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c51f2-c7b9-453d-90fe-313c511b8840 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:15.376928 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:15.400645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c51f2-c7b9-453d-90fe-313c511b8840 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:15.401029 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 819/1644] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:15 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:15.402086 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110758d-8d97-467a-9a5c-ce8c8d4bf1ef None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:15.409643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df800a-4428-44d3-ae49-9522785551cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:15.409954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df800a-4428-44d3-ae49-9522785551cf service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:15.410652 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df800a-4428-44d3-ae49-9522785551cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:15.411008 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df800a-4428-44d3-ae49-9522785551cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:15.421060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110758d-8d97-467a-9a5c-ce8c8d4bf1ef None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:15.425926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df800a-4428-44d3-ae49-9522785551cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:15.427937 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df800a-4428-44d3-ae49-9522785551cf service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:15.431978 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af410fd7-b658-4d4e-ad9a-3c28623d4e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:15.433092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af410fd7-b658-4d4e-ad9a-3c28623d4e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:15.433369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af410fd7-b658-4d4e-ad9a-3c28623d4e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:15.433635 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af410fd7-b658-4d4e-ad9a-3c28623d4e63 admin admin] PATH_INFO: `/v3/projects/2fe434ee7dc74190b66e1875bfa986e8/users/d73c96ae7b444d8287b96b0f24f81d47/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:15.436216 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af410fd7-b658-4d4e-ad9a-3c28623d4e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fe434ee7dc74190b66e1875bfa986e8, user_id=d73c96ae7b444d8287b96b0f24f81d47, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:15.440138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af410fd7-b658-4d4e-ad9a-3c28623d4e63 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:15.470537 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 820/1645] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:15 2022] PUT /identity/v3/projects/2fe434ee7dc74190b66e1875bfa986e8/users/d73c96ae7b444d8287b96b0f24f81d47/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:15.478113 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73dd30-e1b4-44d6-8698-3c4adfd0f808 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:15.481424 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73dd30-e1b4-44d6-8698-3c4adfd0f808 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:15.481832 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73dd30-e1b4-44d6-8698-3c4adfd0f808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:15.509517 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 826/1646] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:15.571520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:15.576968 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-1e73dd30-e1b4-44d6-8698-3c4adfd0f808 None None] MFA Rules not processed for user `d73c96ae7b444d8287b96b0f24f81d47`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:15.578539 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-1e73dd30-e1b4-44d6-8698-3c4adfd0f808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:15.738286 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 821/1647] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:15 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:16.062125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca034cb5-9702-414a-825b-d83338df3df1 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:16.062125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca034cb5-9702-414a-825b-d83338df3df1 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:16.062125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca034cb5-9702-414a-825b-d83338df3df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:16.118531 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-ca034cb5-9702-414a-825b-d83338df3df1 None None] MFA Rules not processed for user `10369664f1914c51893eef1929b7d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:16.118531 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ca034cb5-9702-414a-825b-d83338df3df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:16.185002 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 827/1648] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:16.188318 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744c740-c98c-4f90-8a6d-aa5e48b1eef8 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:16.188609 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744c740-c98c-4f90-8a6d-aa5e48b1eef8 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:16.188894 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744c740-c98c-4f90-8a6d-aa5e48b1eef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:16.213877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-d744c740-c98c-4f90-8a6d-aa5e48b1eef8 None None] MFA Rules not processed for user `10369664f1914c51893eef1929b7d26b`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:16.215022 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d744c740-c98c-4f90-8a6d-aa5e48b1eef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:16.223511 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 822/1649] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:16.471991 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca034cb5-9702-414a-825b-d83338df3df1 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:16.556975 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca034cb5-9702-414a-825b-d83338df3df1 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:16.561154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee225946-95e8-42d3-8fd0-0fa4a4de9513 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:16.561955 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee225946-95e8-42d3-8fd0-0fa4a4de9513 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:16.562347 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee225946-95e8-42d3-8fd0-0fa4a4de9513 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:16.562793 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee225946-95e8-42d3-8fd0-0fa4a4de9513 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:16.728101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee225946-95e8-42d3-8fd0-0fa4a4de9513 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:16.729697 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee225946-95e8-42d3-8fd0-0fa4a4de9513 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:16.743463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 828/1650] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:16.785858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744c740-c98c-4f90-8a6d-aa5e48b1eef8 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:16.792310 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:16.800480 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744c740-c98c-4f90-8a6d-aa5e48b1eef8 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:16.809031 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de1053-7e47-4aec-ae9e-16836616c111 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:16.810099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de1053-7e47-4aec-ae9e-16836616c111 service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:16.811300 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de1053-7e47-4aec-ae9e-16836616c111 service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:16.811733 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de1053-7e47-4aec-ae9e-16836616c111 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:16.822674 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee225946-95e8-42d3-8fd0-0fa4a4de9513 service cinder] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:16.839734 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee225946-95e8-42d3-8fd0-0fa4a4de9513 service cinder] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:16.854285 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b35f8-84c6-47b0-976b-dd00d73c9381 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:16.857195 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b35f8-84c6-47b0-976b-dd00d73c9381 service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:16.857973 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b35f8-84c6-47b0-976b-dd00d73c9381 service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:16.858267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b35f8-84c6-47b0-976b-dd00d73c9381 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:16.881916 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:16.887105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62b35f8-84c6-47b0-976b-dd00d73c9381 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:16.891236 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62b35f8-84c6-47b0-976b-dd00d73c9381 service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:16.914242 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 829/1651] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:50:16.915342 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b35f8-84c6-47b0-976b-dd00d73c9381 service nova] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:16.924468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b35f8-84c6-47b0-976b-dd00d73c9381 service nova] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:16.928253 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7fe34-7505-4b9d-b1c0-fae9b0c31688 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:16.928900 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7fe34-7505-4b9d-b1c0-fae9b0c31688 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:16.929230 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7fe34-7505-4b9d-b1c0-fae9b0c31688 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:16.929508 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7fe34-7505-4b9d-b1c0-fae9b0c31688 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:16.942327 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de1053-7e47-4aec-ae9e-16836616c111 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:16.943775 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7fe34-7505-4b9d-b1c0-fae9b0c31688 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:16.944373 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de1053-7e47-4aec-ae9e-16836616c111 service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:16.946405 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7fe34-7505-4b9d-b1c0-fae9b0c31688 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:16.966084 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 830/1652] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:16 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:16.968698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 823/1653] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:16 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:17.600093 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7fe34-7505-4b9d-b1c0-fae9b0c31688 service cinder] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:17.614233 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7fe34-7505-4b9d-b1c0-fae9b0c31688 service cinder] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:17.619670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3fe4e1-4756-4c9c-a5fe-4747e0242e90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:17.620434 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fe4e1-4756-4c9c-a5fe-4747e0242e90 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:17.620729 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fe4e1-4756-4c9c-a5fe-4747e0242e90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:17.621135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3fe4e1-4756-4c9c-a5fe-4747e0242e90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:17.711888 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3fe4e1-4756-4c9c-a5fe-4747e0242e90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:17.712852 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3fe4e1-4756-4c9c-a5fe-4747e0242e90 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:17.725826 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 831/1654] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:17.911813 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5e5ca-3e3c-4608-b518-94120d9a03a6 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:17.911813 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5e5ca-3e3c-4608-b518-94120d9a03a6 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:17.911813 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd5e5ca-3e3c-4608-b518-94120d9a03a6 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:17.914492 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 824/1655] 10.222.0.13 () {54 vars in 912 bytes} [Fri May 27 19:50:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:50:17.920120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cff49-37df-4e44-899c-197d057a5289 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:17.920740 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cff49-37df-4e44-899c-197d057a5289 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:17.921238 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7cff49-37df-4e44-899c-197d057a5289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:17.947275 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5e5ca-3e3c-4608-b518-94120d9a03a6 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:18.029866 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-4e7cff49-37df-4e44-899c-197d057a5289 None None] MFA Rules not processed for user `dd34f15400c4476bbb2161dbe1b8b2b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:18.031088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-4e7cff49-37df-4e44-899c-197d057a5289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:18.087446 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd5e5ca-3e3c-4608-b518-94120d9a03a6 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:18.097627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7984a5-572a-461c-82ff-ce7914116046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:18.098789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7984a5-572a-461c-82ff-ce7914116046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:18.099312 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7984a5-572a-461c-82ff-ce7914116046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:18.099819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7984a5-572a-461c-82ff-ce7914116046 admin admin] PATH_INFO: `/v3/users/a3ef31e3eac944898442f7a273334ec0` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:18.101392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7984a5-572a-461c-82ff-ce7914116046 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3ef31e3eac944898442f7a273334ec0)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:18.103855 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7984a5-572a-461c-82ff-ce7914116046 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:18.167564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 832/1656] 10.222.0.13 () {58 vars in 1009 bytes} [Fri May 27 19:50:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:50:18.200489 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7cff49-37df-4e44-899c-197d057a5289 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:18.212339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7cff49-37df-4e44-899c-197d057a5289 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:18.228124 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae2d08-a3b5-4162-91d2-17863fa511eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:18.228124 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae2d08-a3b5-4162-91d2-17863fa511eb service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:18.228124 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae2d08-a3b5-4162-91d2-17863fa511eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:18.229199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae2d08-a3b5-4162-91d2-17863fa511eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:18.262145 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae2d08-a3b5-4162-91d2-17863fa511eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:18.263763 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae2d08-a3b5-4162-91d2-17863fa511eb service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:18.292976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 833/1657] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:50:18.384099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-9e7984a5-572a-461c-82ff-ce7914116046 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3ef31e3eac944898442f7a273334ec0'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:18.430832 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-9e7984a5-572a-461c-82ff-ce7914116046 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3ef31e3eac944898442f7a273334ec0'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:18.541142 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-9e7984a5-572a-461c-82ff-ce7914116046 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3ef31e3eac944898442f7a273334ec0'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:18.541793 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-9e7984a5-572a-461c-82ff-ce7914116046 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3ef31e3eac944898442f7a273334ec0'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:18.542487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-9e7984a5-572a-461c-82ff-ce7914116046 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3ef31e3eac944898442f7a273334ec0'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:18.575559 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 825/1658] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:50:17 2022] DELETE /identity/v3/users/a3ef31e3eac944898442f7a273334ec0 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:18.586810 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10833b8-8865-4454-bd52-463355e9abf8 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:18.586810 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10833b8-8865-4454-bd52-463355e9abf8 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:18.586810 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10833b8-8865-4454-bd52-463355e9abf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:18.631949 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-a10833b8-8865-4454-bd52-463355e9abf8 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:18.634155 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-a10833b8-8865-4454-bd52-463355e9abf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:18.763628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 834/1659] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:18.794745 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7984a5-572a-461c-82ff-ce7914116046 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:18.796689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-9e7984a5-572a-461c-82ff-ce7914116046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:18.850429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:18.918753 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7984a5-572a-461c-82ff-ce7914116046 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:18.924391 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2c7a0-27b4-476a-9c99-af7ac3501c46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:18.925735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2c7a0-27b4-476a-9c99-af7ac3501c46 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:18.926195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2c7a0-27b4-476a-9c99-af7ac3501c46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:18.926799 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2c7a0-27b4-476a-9c99-af7ac3501c46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:18.939676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2c7a0-27b4-476a-9c99-af7ac3501c46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:18.941551 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2c7a0-27b4-476a-9c99-af7ac3501c46 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:18.958954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10833b8-8865-4454-bd52-463355e9abf8 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:18.960641 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-a10833b8-8865-4454-bd52-463355e9abf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:18.978788 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 826/1660] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:18.979305 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:18.984923 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10833b8-8865-4454-bd52-463355e9abf8 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:18.994982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a107fd-ab07-4754-a9a7-a79e641f1d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:18.996854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a107fd-ab07-4754-a9a7-a79e641f1d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:18.997402 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a107fd-ab07-4754-a9a7-a79e641f1d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:18.998041 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a107fd-ab07-4754-a9a7-a79e641f1d50 admin admin] PATH_INFO: `/v3/users/acec6981edfe407286b27ad75d42eefc` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:19.000221 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a107fd-ab07-4754-a9a7-a79e641f1d50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acec6981edfe407286b27ad75d42eefc)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:19.001799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: WARNING py.warnings [None req-45a107fd-ab07-4754-a9a7-a79e641f1d50 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 19:50:19.001799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: warnings.warn(msg) May 27 19:50:19.001799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: May 27 19:50:19.004047 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a107fd-ab07-4754-a9a7-a79e641f1d50 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:19.357660 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-45a107fd-ab07-4754-a9a7-a79e641f1d50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acec6981edfe407286b27ad75d42eefc'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:19.418431 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-45a107fd-ab07-4754-a9a7-a79e641f1d50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acec6981edfe407286b27ad75d42eefc'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:19.512012 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-45a107fd-ab07-4754-a9a7-a79e641f1d50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acec6981edfe407286b27ad75d42eefc'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:19.512670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-45a107fd-ab07-4754-a9a7-a79e641f1d50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acec6981edfe407286b27ad75d42eefc'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:19.535960 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-45a107fd-ab07-4754-a9a7-a79e641f1d50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acec6981edfe407286b27ad75d42eefc'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:19.553513 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 835/1661] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:50:18 2022] DELETE /identity/v3/users/acec6981edfe407286b27ad75d42eefc => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:19.565121 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11381d57-8a8a-4522-8ef6-192c4b45efa6 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:19.565421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11381d57-8a8a-4522-8ef6-192c4b45efa6 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:19.565764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11381d57-8a8a-4522-8ef6-192c4b45efa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:19.611204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-11381d57-8a8a-4522-8ef6-192c4b45efa6 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:19.612424 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-11381d57-8a8a-4522-8ef6-192c4b45efa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:19.762181 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 827/1662] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:19.801498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a107fd-ab07-4754-a9a7-a79e641f1d50 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:19.803130 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-45a107fd-ab07-4754-a9a7-a79e641f1d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:19.947249 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a107fd-ab07-4754-a9a7-a79e641f1d50 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:19.952785 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b1e8b-dc49-42a7-b392-c763c6bf0db2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:19.953514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b1e8b-dc49-42a7-b392-c763c6bf0db2 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:19.954120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b1e8b-dc49-42a7-b392-c763c6bf0db2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:19.954319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b1e8b-dc49-42a7-b392-c763c6bf0db2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:19.970253 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b1e8b-dc49-42a7-b392-c763c6bf0db2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:19.971983 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b1e8b-dc49-42a7-b392-c763c6bf0db2 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:19.983657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 836/1663] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:20.035504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f5c53-757e-4a38-8d1d-ac696ac86226 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:20.036863 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f5c53-757e-4a38-8d1d-ac696ac86226 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:20.037362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489f5c53-757e-4a38-8d1d-ac696ac86226 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:20.039372 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 828/1664] 10.222.0.13 () {54 vars in 960 bytes} [Fri May 27 19:50:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:50:20.045279 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb48715-e383-49f7-94c6-c4b270249810 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:20.045871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb48715-e383-49f7-94c6-c4b270249810 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:20.046347 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb48715-e383-49f7-94c6-c4b270249810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:20.094303 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-6cb48715-e383-49f7-94c6-c4b270249810 None None] MFA Rules not processed for user `ffc2ed13a0c84f128b795002892efc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:20.096835 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-6cb48715-e383-49f7-94c6-c4b270249810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:20.280835 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 837/1665] 10.222.0.13 () {58 vars in 1057 bytes} [Fri May 27 19:50:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:20.289786 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f5c53-757e-4a38-8d1d-ac696ac86226 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:20.316879 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489f5c53-757e-4a38-8d1d-ac696ac86226 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:20.325472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb48715-e383-49f7-94c6-c4b270249810 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:20.327408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-6cb48715-e383-49f7-94c6-c4b270249810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:20.346511 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44f5040-73b1-4332-8bea-83a28065beef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2ed13a0c84f128b795002892efc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:20.348813 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f5040-73b1-4332-8bea-83a28065beef service placement] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:20.353478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f5040-73b1-4332-8bea-83a28065beef service placement] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:20.356068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f5040-73b1-4332-8bea-83a28065beef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:20.361714 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-b44f5040-73b1-4332-8bea-83a28065beef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:20.368348 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb48715-e383-49f7-94c6-c4b270249810 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:20.386836 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bfa84f-af9c-4ee7-bcd3-a3741013eabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:20.388211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfa84f-af9c-4ee7-bcd3-a3741013eabd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:20.388211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfa84f-af9c-4ee7-bcd3-a3741013eabd admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:20.388444 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bfa84f-af9c-4ee7-bcd3-a3741013eabd admin admin] PATH_INFO: `/v3/projects/23e6266a43c845acaae1c2c2a1ffbfe8` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:20.394537 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfa84f-af9c-4ee7-bcd3-a3741013eabd admin admin] RBAC: Authorizing `identity:delete_project(project_id=23e6266a43c845acaae1c2c2a1ffbfe8)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:20.396585 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bfa84f-af9c-4ee7-bcd3-a3741013eabd admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:20.404026 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44f5040-73b1-4332-8bea-83a28065beef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:20.409273 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44f5040-73b1-4332-8bea-83a28065beef service placement] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:20.437923 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:20.451121 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 829/1666] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:20.595365 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d9bfa84f-af9c-4ee7-bcd3-a3741013eabd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23e6266a43c845acaae1c2c2a1ffbfe8', 'request_id': 'req-d9bfa84f-af9c-4ee7-bcd3-a3741013eabd'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:20.608510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d9bfa84f-af9c-4ee7-bcd3-a3741013eabd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23e6266a43c845acaae1c2c2a1ffbfe8', 'request_id': 'req-d9bfa84f-af9c-4ee7-bcd3-a3741013eabd'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:20.627819 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d9bfa84f-af9c-4ee7-bcd3-a3741013eabd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23e6266a43c845acaae1c2c2a1ffbfe8', 'request_id': 'req-d9bfa84f-af9c-4ee7-bcd3-a3741013eabd'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:20.708710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d9bfa84f-af9c-4ee7-bcd3-a3741013eabd admin admin] The token cache is being invalidate because project 23e6266a43c845acaae1c2c2a1ffbfe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:50:20.709292 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d9bfa84f-af9c-4ee7-bcd3-a3741013eabd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:20.711359 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d9bfa84f-af9c-4ee7-bcd3-a3741013eabd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:20.714754 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 838/1667] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:50:20 2022] DELETE /identity/v3/projects/23e6266a43c845acaae1c2c2a1ffbfe8 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:20.719433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ae0e5-0cad-418c-88e2-601e364d1191 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:20.719743 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ae0e5-0cad-418c-88e2-601e364d1191 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:20.720026 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ae0e5-0cad-418c-88e2-601e364d1191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:20.768713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-d69ae0e5-0cad-418c-88e2-601e364d1191 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:20.769988 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d69ae0e5-0cad-418c-88e2-601e364d1191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:20.942185 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 830/1668] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:20.950549 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54196bde-81d4-4b3a-bf47-3f7afb8908b2 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:20.951488 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54196bde-81d4-4b3a-bf47-3f7afb8908b2 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:20.951911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54196bde-81d4-4b3a-bf47-3f7afb8908b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:20.995868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-54196bde-81d4-4b3a-bf47-3f7afb8908b2 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:20.997291 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-54196bde-81d4-4b3a-bf47-3f7afb8908b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:21.019463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 839/1669] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:21.024943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ae0e5-0cad-418c-88e2-601e364d1191 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:21.039423 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ae0e5-0cad-418c-88e2-601e364d1191 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:21.055363 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84001b65-9c62-4597-a8e5-8e360b477274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:21.056608 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84001b65-9c62-4597-a8e5-8e360b477274 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:21.057108 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84001b65-9c62-4597-a8e5-8e360b477274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:21.057525 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84001b65-9c62-4597-a8e5-8e360b477274 admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:21.058703 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84001b65-9c62-4597-a8e5-8e360b477274 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:50:21.059287 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84001b65-9c62-4597-a8e5-8e360b477274 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:21.068046 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84001b65-9c62-4597-a8e5-8e360b477274 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:21.120066 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 831/1670] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:50:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:50:21.123281 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54196bde-81d4-4b3a-bf47-3f7afb8908b2 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:21.145153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54196bde-81d4-4b3a-bf47-3f7afb8908b2 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:21.161855 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a2b70-9280-445d-9863-1cde0e92faf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:21.162557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a2b70-9280-445d-9863-1cde0e92faf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:21.163415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a2b70-9280-445d-9863-1cde0e92faf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:21.163707 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a2b70-9280-445d-9863-1cde0e92faf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:21.165042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a2b70-9280-445d-9863-1cde0e92faf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:21.172355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a2b70-9280-445d-9863-1cde0e92faf1 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:21.288914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84001b65-9c62-4597-a8e5-8e360b477274 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:21.291775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-84001b65-9c62-4597-a8e5-8e360b477274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:21.318890 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84001b65-9c62-4597-a8e5-8e360b477274 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:21.337384 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5834dc-8c96-4f5b-b122-3fbbe0d50794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:21.338390 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5834dc-8c96-4f5b-b122-3fbbe0d50794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:21.338717 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5834dc-8c96-4f5b-b122-3fbbe0d50794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:21.339005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5834dc-8c96-4f5b-b122-3fbbe0d50794 admin admin] PATH_INFO: `/v3/projects/0af218a6c1d24642b7948cde9d6fae28` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:21.346379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5834dc-8c96-4f5b-b122-3fbbe0d50794 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0af218a6c1d24642b7948cde9d6fae28)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:21.346897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: WARNING py.warnings [None req-ac5834dc-8c96-4f5b-b122-3fbbe0d50794 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 19:50:21.346897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: warnings.warn(msg) May 27 19:50:21.346897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: May 27 19:50:21.348574 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5834dc-8c96-4f5b-b122-3fbbe0d50794 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:21.398624 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 840/1671] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:21 2022] POST /identity/v3/projects => generated 403 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:21.405412 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a2b70-9280-445d-9863-1cde0e92faf1 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:21.502471 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:21.563912 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-ac5834dc-8c96-4f5b-b122-3fbbe0d50794 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0af218a6c1d24642b7948cde9d6fae28', 'request_id': 'req-ac5834dc-8c96-4f5b-b122-3fbbe0d50794'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:21.566612 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a2b70-9280-445d-9863-1cde0e92faf1 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:21.576563 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-ac5834dc-8c96-4f5b-b122-3fbbe0d50794 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0af218a6c1d24642b7948cde9d6fae28', 'request_id': 'req-ac5834dc-8c96-4f5b-b122-3fbbe0d50794'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:21.653897 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520ddd4-1ad5-4b22-b919-3c32cf158d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:21.655188 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520ddd4-1ad5-4b22-b919-3c32cf158d7a admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:21.655677 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520ddd4-1ad5-4b22-b919-3c32cf158d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:21.656149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520ddd4-1ad5-4b22-b919-3c32cf158d7a admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:21.657082 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520ddd4-1ad5-4b22-b919-3c32cf158d7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:21.659686 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520ddd4-1ad5-4b22-b919-3c32cf158d7a admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:21.662952 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-ac5834dc-8c96-4f5b-b122-3fbbe0d50794 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0af218a6c1d24642b7948cde9d6fae28', 'request_id': 'req-ac5834dc-8c96-4f5b-b122-3fbbe0d50794'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:21.702202 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-ac5834dc-8c96-4f5b-b122-3fbbe0d50794 admin admin] The token cache is being invalidate because project 0af218a6c1d24642b7948cde9d6fae28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:50:21.705993 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-ac5834dc-8c96-4f5b-b122-3fbbe0d50794 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:21.707062 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-ac5834dc-8c96-4f5b-b122-3fbbe0d50794 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:21.712262 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 832/1672] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:50:21 2022] DELETE /identity/v3/projects/0af218a6c1d24642b7948cde9d6fae28 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:21.725441 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5834dc-8c96-4f5b-b122-3fbbe0d50794 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:21.730012 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-ac5834dc-8c96-4f5b-b122-3fbbe0d50794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:21.796739 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 841/1673] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:21 2022] POST /identity/v3/users => generated 302 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:21.799680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5834dc-8c96-4f5b-b122-3fbbe0d50794 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:21.801090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520ddd4-1ad5-4b22-b919-3c32cf158d7a admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:21.806614 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-9520ddd4-1ad5-4b22-b919-3c32cf158d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:21.809688 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7f8f7-9c69-4567-ad12-fb36e89c23fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:21.824159 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7f8f7-9c69-4567-ad12-fb36e89c23fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:21.824456 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7f8f7-9c69-4567-ad12-fb36e89c23fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:21.828093 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7f8f7-9c69-4567-ad12-fb36e89c23fe admin admin] PATH_INFO: `/v3/users/eb9a5b210d504772b120e868c42d9ece` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:21.830396 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7f8f7-9c69-4567-ad12-fb36e89c23fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb9a5b210d504772b120e868c42d9ece)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:21.834434 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520ddd4-1ad5-4b22-b919-3c32cf158d7a admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:21.840513 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7f8f7-9c69-4567-ad12-fb36e89c23fe admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:21.845729 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf97248-e498-46c3-bca3-f903756d77db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:21.846400 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf97248-e498-46c3-bca3-f903756d77db admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:21.846743 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf97248-e498-46c3-bca3-f903756d77db admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:21.847079 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf97248-e498-46c3-bca3-f903756d77db admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:21.854226 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf97248-e498-46c3-bca3-f903756d77db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:21.854226 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf97248-e498-46c3-bca3-f903756d77db admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:21.872070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 842/1674] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:21.875891 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf97248-e498-46c3-bca3-f903756d77db admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:21.896131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf97248-e498-46c3-bca3-f903756d77db admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:21.908268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a25c1-18d8-4106-88ed-9b350e9459b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:21.909347 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a25c1-18d8-4106-88ed-9b350e9459b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:21.909594 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a25c1-18d8-4106-88ed-9b350e9459b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:21.909850 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a25c1-18d8-4106-88ed-9b350e9459b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:21.910591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a25c1-18d8-4106-88ed-9b350e9459b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:21.912212 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a25c1-18d8-4106-88ed-9b350e9459b0 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:21.939670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 843/1675] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:21 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:21.941909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a25c1-18d8-4106-88ed-9b350e9459b0 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:21.960734 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a25c1-18d8-4106-88ed-9b350e9459b0 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:21.982877 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420e70a-15e6-4610-bffe-30af69926b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:21.985229 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420e70a-15e6-4610-bffe-30af69926b48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:21.985654 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420e70a-15e6-4610-bffe-30af69926b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:21.986989 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420e70a-15e6-4610-bffe-30af69926b48 admin admin] PATH_INFO: `/v3/projects/7c727f840dd646639798a916ff118e33/users/c511d932107a41daab469b58d4ee9d60/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:22.032790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e420e70a-15e6-4610-bffe-30af69926b48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c727f840dd646639798a916ff118e33, user_id=c511d932107a41daab469b58d4ee9d60, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:22.037457 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e420e70a-15e6-4610-bffe-30af69926b48 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:22.064678 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:22.151809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-73a7f8f7-9c69-4567-ad12-fb36e89c23fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb9a5b210d504772b120e868c42d9ece'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:22.187383 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 844/1676] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:21 2022] PUT /identity/v3/projects/7c727f840dd646639798a916ff118e33/users/c511d932107a41daab469b58d4ee9d60/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:22.192706 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a30110-7188-4d70-9abe-dfe9f6b8cb19 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:22.193401 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a30110-7188-4d70-9abe-dfe9f6b8cb19 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:22.194407 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-73a7f8f7-9c69-4567-ad12-fb36e89c23fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb9a5b210d504772b120e868c42d9ece'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:22.194538 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a30110-7188-4d70-9abe-dfe9f6b8cb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:22.249749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-35a30110-7188-4d70-9abe-dfe9f6b8cb19 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:22.253942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-35a30110-7188-4d70-9abe-dfe9f6b8cb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:22.303058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-73a7f8f7-9c69-4567-ad12-fb36e89c23fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb9a5b210d504772b120e868c42d9ece'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:22.304400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-73a7f8f7-9c69-4567-ad12-fb36e89c23fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb9a5b210d504772b120e868c42d9ece'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:22.306037 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-73a7f8f7-9c69-4567-ad12-fb36e89c23fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb9a5b210d504772b120e868c42d9ece'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:22.371571 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 833/1677] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:50:21 2022] DELETE /identity/v3/users/eb9a5b210d504772b120e868c42d9ece => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:22.412314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 845/1678] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:22.581331 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7f8f7-9c69-4567-ad12-fb36e89c23fe admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:22.582908 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-73a7f8f7-9c69-4567-ad12-fb36e89c23fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:22.725751 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7f8f7-9c69-4567-ad12-fb36e89c23fe admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:22.733024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60434081-6d7d-4279-a8c9-39bcff48f84e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:22.733930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60434081-6d7d-4279-a8c9-39bcff48f84e service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:22.734315 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60434081-6d7d-4279-a8c9-39bcff48f84e service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:22.734764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60434081-6d7d-4279-a8c9-39bcff48f84e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:22.872906 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60434081-6d7d-4279-a8c9-39bcff48f84e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:22.875152 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60434081-6d7d-4279-a8c9-39bcff48f84e service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:22.919232 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 834/1679] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:23.060762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e67695-4090-4abb-a69e-1de506f53ad2 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:23.061819 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e67695-4090-4abb-a69e-1de506f53ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:23.062464 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e67695-4090-4abb-a69e-1de506f53ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:23.095734 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-f0e67695-4090-4abb-a69e-1de506f53ad2 None None] MFA Rules not processed for user `ed8754ddcfd54c24ac923db0fedc1606`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:23.097513 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-f0e67695-4090-4abb-a69e-1de506f53ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:23.138075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:23.183601 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 846/1680] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:23.200041 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60434081-6d7d-4279-a8c9-39bcff48f84e service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:23.204415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-60434081-6d7d-4279-a8c9-39bcff48f84e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:23.207442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e67695-4090-4abb-a69e-1de506f53ad2 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:23.209004 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-f0e67695-4090-4abb-a69e-1de506f53ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:23.233070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e67695-4090-4abb-a69e-1de506f53ad2 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:23.245289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5851542-3829-4563-930b-df8b35c78875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:23.246422 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5851542-3829-4563-930b-df8b35c78875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:23.246876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5851542-3829-4563-930b-df8b35c78875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:23.247285 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5851542-3829-4563-930b-df8b35c78875 admin admin] PATH_INFO: `/v3/projects/b0106f670f1240ad96c504fed8bb5fb8` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:23.249748 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5851542-3829-4563-930b-df8b35c78875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0106f670f1240ad96c504fed8bb5fb8)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:23.251713 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5851542-3829-4563-930b-df8b35c78875 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:23.342871 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60434081-6d7d-4279-a8c9-39bcff48f84e service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:23.360524 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab815e2a-0eb4-4b0a-8457-c06ef84175ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:23.362081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab815e2a-0eb4-4b0a-8457-c06ef84175ff service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:23.362379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab815e2a-0eb4-4b0a-8457-c06ef84175ff service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:23.362727 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab815e2a-0eb4-4b0a-8457-c06ef84175ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:23.372032 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-b5851542-3829-4563-930b-df8b35c78875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0106f670f1240ad96c504fed8bb5fb8', 'request_id': 'req-b5851542-3829-4563-930b-df8b35c78875'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:23.376321 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab815e2a-0eb4-4b0a-8457-c06ef84175ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:23.380446 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab815e2a-0eb4-4b0a-8457-c06ef84175ff service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:23.381134 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-b5851542-3829-4563-930b-df8b35c78875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0106f670f1240ad96c504fed8bb5fb8', 'request_id': 'req-b5851542-3829-4563-930b-df8b35c78875'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:23.389789 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-b5851542-3829-4563-930b-df8b35c78875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0106f670f1240ad96c504fed8bb5fb8', 'request_id': 'req-b5851542-3829-4563-930b-df8b35c78875'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:23.401937 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 835/1681] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:23.528267 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab815e2a-0eb4-4b0a-8457-c06ef84175ff service nova] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:23.529891 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-ab815e2a-0eb4-4b0a-8457-c06ef84175ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:23.533623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-b5851542-3829-4563-930b-df8b35c78875 admin admin] The token cache is being invalidate because project b0106f670f1240ad96c504fed8bb5fb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:50:23.535511 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-b5851542-3829-4563-930b-df8b35c78875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:23.539771 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-b5851542-3829-4563-930b-df8b35c78875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:23.557987 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 847/1682] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:50:23 2022] DELETE /identity/v3/projects/b0106f670f1240ad96c504fed8bb5fb8 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:23.567592 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a02e7e-2389-4fba-a5d4-045171dc2833 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:23.568090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a02e7e-2389-4fba-a5d4-045171dc2833 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:23.568514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a02e7e-2389-4fba-a5d4-045171dc2833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:23.615061 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-95a02e7e-2389-4fba-a5d4-045171dc2833 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:23.615773 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-95a02e7e-2389-4fba-a5d4-045171dc2833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:23.743583 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab815e2a-0eb4-4b0a-8457-c06ef84175ff service nova] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:23.764523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 848/1683] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:23.772641 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae0719-2945-4f6c-b7d5-57d454a1a1ee None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:23.772940 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae0719-2945-4f6c-b7d5-57d454a1a1ee None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:23.773781 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae0719-2945-4f6c-b7d5-57d454a1a1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:23.814692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-60ae0719-2945-4f6c-b7d5-57d454a1a1ee None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:23.815933 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-60ae0719-2945-4f6c-b7d5-57d454a1a1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:23.836355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 849/1684] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:50:23.840324 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae0719-2945-4f6c-b7d5-57d454a1a1ee None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:23.859756 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ae0719-2945-4f6c-b7d5-57d454a1a1ee None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:23.863275 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed912d-74dd-4edc-9a79-aaee7abaa0f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:23.864358 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed912d-74dd-4edc-9a79-aaee7abaa0f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:23.864836 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed912d-74dd-4edc-9a79-aaee7abaa0f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:23.865279 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ed912d-74dd-4edc-9a79-aaee7abaa0f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:23.869785 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d2ed912d-74dd-4edc-9a79-aaee7abaa0f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:23.873797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24384af7-1343-4c35-9e35-627f41e71c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:23.874491 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24384af7-1343-4c35-9e35-627f41e71c82 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:23.874776 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24384af7-1343-4c35-9e35-627f41e71c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:23.875055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24384af7-1343-4c35-9e35-627f41e71c82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:23.875881 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24384af7-1343-4c35-9e35-627f41e71c82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:50:23.876193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24384af7-1343-4c35-9e35-627f41e71c82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:23.881974 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24384af7-1343-4c35-9e35-627f41e71c82 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:23.910641 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 850/1685] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:50:23.915430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24384af7-1343-4c35-9e35-627f41e71c82 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:23.940267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24384af7-1343-4c35-9e35-627f41e71c82 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:23.947258 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4681b8-def6-49f7-9a64-74f20ddff49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:23.947954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4681b8-def6-49f7-9a64-74f20ddff49d admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:23.948242 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4681b8-def6-49f7-9a64-74f20ddff49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:23.948524 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4681b8-def6-49f7-9a64-74f20ddff49d admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:23.949262 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4681b8-def6-49f7-9a64-74f20ddff49d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:23.955937 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4681b8-def6-49f7-9a64-74f20ddff49d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:24.052702 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ed912d-74dd-4edc-9a79-aaee7abaa0f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:24.056424 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ed912d-74dd-4edc-9a79-aaee7abaa0f8 service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:24.077546 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 851/1686] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:23 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:24.081550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4681b8-def6-49f7-9a64-74f20ddff49d admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:24.084608 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 836/1687] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:50:24.210387 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4681b8-def6-49f7-9a64-74f20ddff49d admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:24.216331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c23353-ae9e-45c4-b66e-abff71abcb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:24.217264 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c23353-ae9e-45c4-b66e-abff71abcb0b admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:24.217621 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c23353-ae9e-45c4-b66e-abff71abcb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:24.218061 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c23353-ae9e-45c4-b66e-abff71abcb0b admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:24.219092 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c23353-ae9e-45c4-b66e-abff71abcb0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:24.221289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c23353-ae9e-45c4-b66e-abff71abcb0b admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:24.235400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed912d-74dd-4edc-9a79-aaee7abaa0f8 service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:24.235400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d2ed912d-74dd-4edc-9a79-aaee7abaa0f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:24.253461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ed912d-74dd-4edc-9a79-aaee7abaa0f8 service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:24.269881 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a3043-b01d-437b-a8a6-af861d5d9b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:24.270784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a3043-b01d-437b-a8a6-af861d5d9b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:24.271071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a3043-b01d-437b-a8a6-af861d5d9b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:24.271349 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a3043-b01d-437b-a8a6-af861d5d9b2e admin admin] PATH_INFO: `/v3/users/814653f467b14218849d4b7135da0f08` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:24.277815 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a3043-b01d-437b-a8a6-af861d5d9b2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=814653f467b14218849d4b7135da0f08)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:24.280664 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a3043-b01d-437b-a8a6-af861d5d9b2e admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:24.332649 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 852/1688] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:24 2022] POST /identity/v3/users => generated 294 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:50:24.340655 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c23353-ae9e-45c4-b66e-abff71abcb0b admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:24.365276 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c23353-ae9e-45c4-b66e-abff71abcb0b admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:24.382546 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d067409f-f07d-4863-be42-8287fd6dea70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:24.386932 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067409f-f07d-4863-be42-8287fd6dea70 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:24.387225 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067409f-f07d-4863-be42-8287fd6dea70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:24.389689 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d067409f-f07d-4863-be42-8287fd6dea70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:24.393554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d067409f-f07d-4863-be42-8287fd6dea70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:24.398205 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d067409f-f07d-4863-be42-8287fd6dea70 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:24.436901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 853/1689] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:24.438324 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cc3f9-6947-4780-93f7-98e493e01eb0 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:24.438621 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cc3f9-6947-4780-93f7-98e493e01eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:24.438906 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905cc3f9-6947-4780-93f7-98e493e01eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:24.465751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:24.527037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-905cc3f9-6947-4780-93f7-98e493e01eb0 None None] MFA Rules not processed for user `3c9aa6527c13466bbe5dce68a3b81ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:24.527037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-905cc3f9-6947-4780-93f7-98e493e01eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:24.593581 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-7a4a3043-b01d-437b-a8a6-af861d5d9b2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '814653f467b14218849d4b7135da0f08'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:24.651693 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-7a4a3043-b01d-437b-a8a6-af861d5d9b2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '814653f467b14218849d4b7135da0f08'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:24.659747 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 854/1690] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:24.659747 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905cc3f9-6947-4780-93f7-98e493e01eb0 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:24.674609 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905cc3f9-6947-4780-93f7-98e493e01eb0 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:24.691936 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961db32-6e44-465a-820f-aca93ecd85f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:24.692757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961db32-6e44-465a-820f-aca93ecd85f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:24.693139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961db32-6e44-465a-820f-aca93ecd85f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:24.693510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1961db32-6e44-465a-820f-aca93ecd85f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:24.694573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1961db32-6e44-465a-820f-aca93ecd85f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:24.698051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1961db32-6e44-465a-820f-aca93ecd85f0 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:24.741855 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 855/1691] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:24.743708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961db32-6e44-465a-820f-aca93ecd85f0 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:24.749800 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-1961db32-6e44-465a-820f-aca93ecd85f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:24.811599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-7a4a3043-b01d-437b-a8a6-af861d5d9b2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '814653f467b14218849d4b7135da0f08'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:24.812223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-7a4a3043-b01d-437b-a8a6-af861d5d9b2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '814653f467b14218849d4b7135da0f08'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:24.813043 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-7a4a3043-b01d-437b-a8a6-af861d5d9b2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '814653f467b14218849d4b7135da0f08'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:24.830977 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 837/1692] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:50:24 2022] DELETE /identity/v3/users/814653f467b14218849d4b7135da0f08 => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:50:24.831977 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a3043-b01d-437b-a8a6-af861d5d9b2e admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:24.834718 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-7a4a3043-b01d-437b-a8a6-af861d5d9b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:24.862137 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1961db32-6e44-465a-820f-aca93ecd85f0 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:24.937831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a3043-b01d-437b-a8a6-af861d5d9b2e admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:24.946537 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfd514e-4fc7-499c-a4ca-3b87e098b4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:24.947984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd514e-4fc7-499c-a4ca-3b87e098b4d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:24.948286 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd514e-4fc7-499c-a4ca-3b87e098b4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:24.948570 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfd514e-4fc7-499c-a4ca-3b87e098b4d3 admin admin] PATH_INFO: `/v3/projects/bffe93747b8745d587fdc5a26781d993/users/ef08f54dbf9a4f7f962361d580aaa8bc/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:24.976072 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd514e-4fc7-499c-a4ca-3b87e098b4d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bffe93747b8745d587fdc5a26781d993, user_id=ef08f54dbf9a4f7f962361d580aaa8bc, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:24.977103 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fae81-16cc-4131-ba05-1fecfbcf43ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:24.977930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fae81-16cc-4131-ba05-1fecfbcf43ce service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:24.978281 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fae81-16cc-4131-ba05-1fecfbcf43ce service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:24.978652 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fae81-16cc-4131-ba05-1fecfbcf43ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:24.980239 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfd514e-4fc7-499c-a4ca-3b87e098b4d3 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:24.980686 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d80fae81-16cc-4131-ba05-1fecfbcf43ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:25.064255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 838/1693] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:24 2022] PUT /identity/v3/projects/bffe93747b8745d587fdc5a26781d993/users/ef08f54dbf9a4f7f962361d580aaa8bc/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:25.065584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6e641-f8c8-4743-aa73-f070b19c48b4 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:25.066178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6e641-f8c8-4743-aa73-f070b19c48b4 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:25.066460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b6e641-f8c8-4743-aa73-f070b19c48b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:25.067393 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fae81-16cc-4131-ba05-1fecfbcf43ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:25.069041 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fae81-16cc-4131-ba05-1fecfbcf43ce service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:25.105356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 856/1694] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:25.106836 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d95ca6-b796-4c61-9b94-a6e0499e638a None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:25.107132 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d95ca6-b796-4c61-9b94-a6e0499e638a None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:25.109560 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d95ca6-b796-4c61-9b94-a6e0499e638a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:25.119401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-18b6e641-f8c8-4743-aa73-f070b19c48b4 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:25.124510 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-18b6e641-f8c8-4743-aa73-f070b19c48b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:25.225326 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-77d95ca6-b796-4c61-9b94-a6e0499e638a None None] MFA Rules not processed for user `ef08f54dbf9a4f7f962361d580aaa8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:25.228111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-77d95ca6-b796-4c61-9b94-a6e0499e638a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:25.307718 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 839/1695] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:25.309594 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22c95a-2a30-46de-8e5a-e43661019add None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:25.309890 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22c95a-2a30-46de-8e5a-e43661019add None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:25.310173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc22c95a-2a30-46de-8e5a-e43661019add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:25.434665 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-bc22c95a-2a30-46de-8e5a-e43661019add None None] MFA Rules not processed for user `c511d932107a41daab469b58d4ee9d60`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:25.436494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:25.441385 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-bc22c95a-2a30-46de-8e5a-e43661019add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:25.447901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 857/1696] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:25.448750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d95ca6-b796-4c61-9b94-a6e0499e638a None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:25.450208 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-77d95ca6-b796-4c61-9b94-a6e0499e638a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:25.616789 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d95ca6-b796-4c61-9b94-a6e0499e638a None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:25.628864 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50c21a-6ce2-4500-ae21-5cd5754b8697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:25.629590 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50c21a-6ce2-4500-ae21-5cd5754b8697 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:25.629883 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50c21a-6ce2-4500-ae21-5cd5754b8697 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:25.630168 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e50c21a-6ce2-4500-ae21-5cd5754b8697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:25.647006 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 840/1697] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:25.653303 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22c95a-2a30-46de-8e5a-e43661019add None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:25.667880 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc22c95a-2a30-46de-8e5a-e43661019add None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:25.669035 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50c21a-6ce2-4500-ae21-5cd5754b8697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:25.671892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e50c21a-6ce2-4500-ae21-5cd5754b8697 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:25.677850 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4d721-04cf-4ce9-9f82-70458191a8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:25.677850 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4d721-04cf-4ce9-9f82-70458191a8dd admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:25.677850 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4d721-04cf-4ce9-9f82-70458191a8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:25.677850 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c4d721-04cf-4ce9-9f82-70458191a8dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:25.678517 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4d721-04cf-4ce9-9f82-70458191a8dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:25.685039 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c4d721-04cf-4ce9-9f82-70458191a8dd admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:25.714724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 858/1698] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:25.719941 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50c21a-6ce2-4500-ae21-5cd5754b8697 service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:25.727805 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-8e50c21a-6ce2-4500-ae21-5cd5754b8697 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:25.754125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e50c21a-6ce2-4500-ae21-5cd5754b8697 service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:25.767230 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9abaa-a6d8-4875-9e09-ea1e47e72770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:25.768031 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9abaa-a6d8-4875-9e09-ea1e47e72770 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:25.768423 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9abaa-a6d8-4875-9e09-ea1e47e72770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:25.768815 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9abaa-a6d8-4875-9e09-ea1e47e72770 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:25.769742 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c9abaa-a6d8-4875-9e09-ea1e47e72770 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:25.777990 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c9abaa-a6d8-4875-9e09-ea1e47e72770 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:25.831844 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 841/1699] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:25 2022] POST /identity/v3/projects => generated 387 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:25.836405 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4d721-04cf-4ce9-9f82-70458191a8dd admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:25.881758 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 859/1700] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:25 2022] POST /identity/v3/projects => generated 403 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:25.885835 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9abaa-a6d8-4875-9e09-ea1e47e72770 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:25.938755 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c4d721-04cf-4ce9-9f82-70458191a8dd admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:25.984816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9abaa-a6d8-4875-9e09-ea1e47e72770 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:25.996226 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919cc62-e48f-4351-af38-7266087fbb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:25.997198 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919cc62-e48f-4351-af38-7266087fbb17 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:25.998663 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919cc62-e48f-4351-af38-7266087fbb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:25.998953 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919cc62-e48f-4351-af38-7266087fbb17 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:26.000800 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919cc62-e48f-4351-af38-7266087fbb17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:26.005984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d919cc62-e48f-4351-af38-7266087fbb17 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:26.069543 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:26.098753 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26073a6d-6b35-4797-81b2-6b928d0fd872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:26.099506 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26073a6d-6b35-4797-81b2-6b928d0fd872 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:26.099794 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26073a6d-6b35-4797-81b2-6b928d0fd872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:26.100070 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26073a6d-6b35-4797-81b2-6b928d0fd872 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:26.100793 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26073a6d-6b35-4797-81b2-6b928d0fd872 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:26.103341 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26073a6d-6b35-4797-81b2-6b928d0fd872 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:26.147296 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 860/1701] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:25 2022] POST /identity/v3/users => generated 308 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:50:26.153999 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919cc62-e48f-4351-af38-7266087fbb17 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:26.185309 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d919cc62-e48f-4351-af38-7266087fbb17 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:26.201685 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c5023-2d61-4f07-8c9f-806fb73b0d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:26.202700 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c5023-2d61-4f07-8c9f-806fb73b0d61 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:26.203504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c5023-2d61-4f07-8c9f-806fb73b0d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:26.204715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c5023-2d61-4f07-8c9f-806fb73b0d61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:26.205711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c5023-2d61-4f07-8c9f-806fb73b0d61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:26.210607 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c5023-2d61-4f07-8c9f-806fb73b0d61 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:26.236542 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 861/1702] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:26.237855 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c5023-2d61-4f07-8c9f-806fb73b0d61 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:26.252345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c5023-2d61-4f07-8c9f-806fb73b0d61 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:26.264848 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863d38d-51f6-42ba-aa32-ef07a5869731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:26.266883 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863d38d-51f6-42ba-aa32-ef07a5869731 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:26.267441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863d38d-51f6-42ba-aa32-ef07a5869731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:26.267874 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863d38d-51f6-42ba-aa32-ef07a5869731 admin admin] PATH_INFO: `/v3/projects/c95c573fbebf47c192b176d307c2b01e/users/84c7224a6592470a9c873804361d5538/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:26.300601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 842/1703] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:25 2022] POST /identity/v3/users => generated 300 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:26.301941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26073a6d-6b35-4797-81b2-6b928d0fd872 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:26.324243 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26073a6d-6b35-4797-81b2-6b928d0fd872 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:26.330696 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863d38d-51f6-42ba-aa32-ef07a5869731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c95c573fbebf47c192b176d307c2b01e, user_id=84c7224a6592470a9c873804361d5538, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:26.333539 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863d38d-51f6-42ba-aa32-ef07a5869731 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:26.339556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20b7ba-d078-4484-b642-ca34112d143d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:26.340346 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20b7ba-d078-4484-b642-ca34112d143d admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:26.340701 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20b7ba-d078-4484-b642-ca34112d143d admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:26.341078 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f20b7ba-d078-4484-b642-ca34112d143d admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:26.341890 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20b7ba-d078-4484-b642-ca34112d143d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:26.348639 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f20b7ba-d078-4484-b642-ca34112d143d admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:26.359056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:26.381410 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 843/1704] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:26.386338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20b7ba-d078-4484-b642-ca34112d143d admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:26.400512 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f20b7ba-d078-4484-b642-ca34112d143d admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:26.406777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e537a5-43f9-4979-96fe-502df110fa67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:26.408375 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e537a5-43f9-4979-96fe-502df110fa67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:26.408831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e537a5-43f9-4979-96fe-502df110fa67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:26.409293 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e537a5-43f9-4979-96fe-502df110fa67 admin admin] PATH_INFO: `/v3/projects/5e3d495d42d8454391c8a81d1383d74c/users/e491f573c643449da845954bc5b659a8/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:26.457707 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e537a5-43f9-4979-96fe-502df110fa67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e3d495d42d8454391c8a81d1383d74c, user_id=e491f573c643449da845954bc5b659a8, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:26.461437 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e537a5-43f9-4979-96fe-502df110fa67 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:26.479059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 862/1705] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:26 2022] PUT /identity/v3/projects/c95c573fbebf47c192b176d307c2b01e/users/84c7224a6592470a9c873804361d5538/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:26.480054 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863d38d-51f6-42ba-aa32-ef07a5869731 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:26.606909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 844/1706] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:26 2022] PUT /identity/v3/projects/5e3d495d42d8454391c8a81d1383d74c/users/e491f573c643449da845954bc5b659a8/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:26.611361 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e537a5-43f9-4979-96fe-502df110fa67 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:26.643502 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863d38d-51f6-42ba-aa32-ef07a5869731 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:26.763831 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797bce44-98f9-4d9e-a4db-0eaf34a01d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:26.764543 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797bce44-98f9-4d9e-a4db-0eaf34a01d66 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:26.764826 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797bce44-98f9-4d9e-a4db-0eaf34a01d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:26.765140 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797bce44-98f9-4d9e-a4db-0eaf34a01d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:26.765888 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797bce44-98f9-4d9e-a4db-0eaf34a01d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:26.767775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e537a5-43f9-4979-96fe-502df110fa67 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:26.768729 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797bce44-98f9-4d9e-a4db-0eaf34a01d66 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:26.785111 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639ba7c-55a6-44af-8c65-ff0b68f28ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:26.785807 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639ba7c-55a6-44af-8c65-ff0b68f28ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:26.786090 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639ba7c-55a6-44af-8c65-ff0b68f28ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:26.786370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639ba7c-55a6-44af-8c65-ff0b68f28ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:26.788567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639ba7c-55a6-44af-8c65-ff0b68f28ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:26.790436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e639ba7c-55a6-44af-8c65-ff0b68f28ba1 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:26.811022 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 863/1707] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:26.813354 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797bce44-98f9-4d9e-a4db-0eaf34a01d66 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:26.818113 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 845/1708] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:26.825627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639ba7c-55a6-44af-8c65-ff0b68f28ba1 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:26.850096 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797bce44-98f9-4d9e-a4db-0eaf34a01d66 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:26.856275 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639ba7c-55a6-44af-8c65-ff0b68f28ba1 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:26.867626 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b23d7-7a30-4581-81f9-8fef375b3c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:26.868555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b23d7-7a30-4581-81f9-8fef375b3c86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:26.872021 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b23d7-7a30-4581-81f9-8fef375b3c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:26.872479 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220b23d7-7a30-4581-81f9-8fef375b3c86 admin admin] PATH_INFO: `/v3/projects/c95c573fbebf47c192b176d307c2b01e/users/84c7224a6592470a9c873804361d5538/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:26.882374 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b23d7-7a30-4581-81f9-8fef375b3c86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c95c573fbebf47c192b176d307c2b01e, user_id=84c7224a6592470a9c873804361d5538, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:26.885441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220b23d7-7a30-4581-81f9-8fef375b3c86 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:26.885878 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc237783-e722-47f0-b69b-322b1d4e714f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:26.887155 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc237783-e722-47f0-b69b-322b1d4e714f admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:26.887611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc237783-e722-47f0-b69b-322b1d4e714f admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:26.888026 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc237783-e722-47f0-b69b-322b1d4e714f admin admin] PATH_INFO: `/v3/projects/5e3d495d42d8454391c8a81d1383d74c/users/e491f573c643449da845954bc5b659a8/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:26.892099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc237783-e722-47f0-b69b-322b1d4e714f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e3d495d42d8454391c8a81d1383d74c, user_id=e491f573c643449da845954bc5b659a8, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:26.895042 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc237783-e722-47f0-b69b-322b1d4e714f admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:26.930670 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 846/1709] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:26 2022] PUT /identity/v3/projects/5e3d495d42d8454391c8a81d1383d74c/users/e491f573c643449da845954bc5b659a8/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:26.932310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 864/1710] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:26 2022] PUT /identity/v3/projects/c95c573fbebf47c192b176d307c2b01e/users/84c7224a6592470a9c873804361d5538/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:26.955362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb184b2-0855-4cc3-8315-0cba5f4404a4 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:26.955654 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb184b2-0855-4cc3-8315-0cba5f4404a4 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:26.955943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb184b2-0855-4cc3-8315-0cba5f4404a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:27.020361 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-acb184b2-0855-4cc3-8315-0cba5f4404a4 None None] MFA Rules not processed for user `e491f573c643449da845954bc5b659a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:27.021626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-acb184b2-0855-4cc3-8315-0cba5f4404a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:27.043689 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b23d7-7a30-4581-81f9-8fef375b3c86 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:27.043689 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-220b23d7-7a30-4581-81f9-8fef375b3c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:27.158633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 847/1711] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:27.171329 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb184b2-0855-4cc3-8315-0cba5f4404a4 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:27.172663 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-acb184b2-0855-4cc3-8315-0cba5f4404a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:27.200537 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220b23d7-7a30-4581-81f9-8fef375b3c86 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:27.220644 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073b722-5115-432a-8cb9-0640bd297adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:27.221819 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073b722-5115-432a-8cb9-0640bd297adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:27.222276 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073b722-5115-432a-8cb9-0640bd297adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:27.222699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073b722-5115-432a-8cb9-0640bd297adb admin admin] PATH_INFO: `/v3/projects/7241472ff5c44725bda3e741d79d474d` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:27.225035 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c073b722-5115-432a-8cb9-0640bd297adb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7241472ff5c44725bda3e741d79d474d)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:27.228242 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c073b722-5115-432a-8cb9-0640bd297adb admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:27.296920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb184b2-0855-4cc3-8315-0cba5f4404a4 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:27.301901 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4fea2-2b9e-468b-afd7-1fd8c0e80373 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:27.302851 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4fea2-2b9e-468b-afd7-1fd8c0e80373 service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:27.303657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4fea2-2b9e-468b-afd7-1fd8c0e80373 service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:27.304120 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc4fea2-2b9e-468b-afd7-1fd8c0e80373 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:27.328538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4fea2-2b9e-468b-afd7-1fd8c0e80373 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:27.334239 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc4fea2-2b9e-468b-afd7-1fd8c0e80373 service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:27.383561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 848/1712] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:27 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:50:27.431190 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4fea2-2b9e-468b-afd7-1fd8c0e80373 service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:27.431364 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c073b722-5115-432a-8cb9-0640bd297adb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7241472ff5c44725bda3e741d79d474d', 'request_id': 'req-c073b722-5115-432a-8cb9-0640bd297adb'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:27.438773 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c073b722-5115-432a-8cb9-0640bd297adb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7241472ff5c44725bda3e741d79d474d', 'request_id': 'req-c073b722-5115-432a-8cb9-0640bd297adb'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:27.453755 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc4fea2-2b9e-468b-afd7-1fd8c0e80373 service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:27.455076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c073b722-5115-432a-8cb9-0640bd297adb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7241472ff5c44725bda3e741d79d474d', 'request_id': 'req-c073b722-5115-432a-8cb9-0640bd297adb'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:27.547124 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b905dc-2742-4747-a377-2c59f12a9f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:27.548771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b905dc-2742-4747-a377-2c59f12a9f7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:27.549352 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b905dc-2742-4747-a377-2c59f12a9f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:27.549820 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b905dc-2742-4747-a377-2c59f12a9f7d admin admin] PATH_INFO: `/v3/users/ef08f54dbf9a4f7f962361d580aaa8bc` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:27.553577 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b905dc-2742-4747-a377-2c59f12a9f7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef08f54dbf9a4f7f962361d580aaa8bc)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:27.557035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b905dc-2742-4747-a377-2c59f12a9f7d admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:27.561151 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c073b722-5115-432a-8cb9-0640bd297adb admin admin] The token cache is being invalidate because project 7241472ff5c44725bda3e741d79d474d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:50:27.561452 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c073b722-5115-432a-8cb9-0640bd297adb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:27.569921 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c073b722-5115-432a-8cb9-0640bd297adb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:27.574314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 865/1713] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:50:27 2022] DELETE /identity/v3/projects/7241472ff5c44725bda3e741d79d474d => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:27.579725 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedb9c4-f087-4e0a-af18-1231b0652ac9 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:27.580031 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedb9c4-f087-4e0a-af18-1231b0652ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:27.580321 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedb9c4-f087-4e0a-af18-1231b0652ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:27.611468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-dcedb9c4-f087-4e0a-af18-1231b0652ac9 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:27.623299 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-dcedb9c4-f087-4e0a-af18-1231b0652ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:27.631831 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 866/1714] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:27.640851 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e77a7-7ba4-41b4-971f-4465d80195c7 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:27.641289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e77a7-7ba4-41b4-971f-4465d80195c7 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:27.641640 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e77a7-7ba4-41b4-971f-4465d80195c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:27.663733 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-8b6e77a7-7ba4-41b4-971f-4465d80195c7 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:27.665740 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-8b6e77a7-7ba4-41b4-971f-4465d80195c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:27.678124 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 867/1715] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:27.682250 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e77a7-7ba4-41b4-971f-4465d80195c7 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:27.692162 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e77a7-7ba4-41b4-971f-4465d80195c7 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:27.698330 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da53018-c1ba-4ffe-95ca-f96e050acf96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:27.699014 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da53018-c1ba-4ffe-95ca-f96e050acf96 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:27.699301 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da53018-c1ba-4ffe-95ca-f96e050acf96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:27.699589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da53018-c1ba-4ffe-95ca-f96e050acf96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:27.700806 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da53018-c1ba-4ffe-95ca-f96e050acf96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:50:27.700806 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da53018-c1ba-4ffe-95ca-f96e050acf96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:27.702414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da53018-c1ba-4ffe-95ca-f96e050acf96 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:27.719571 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 868/1716] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:50:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:50:27.720632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da53018-c1ba-4ffe-95ca-f96e050acf96 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:27.731054 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da53018-c1ba-4ffe-95ca-f96e050acf96 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:27.735758 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65affc74-5813-4a18-9b27-351572986bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:27.736422 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65affc74-5813-4a18-9b27-351572986bb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:27.736709 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65affc74-5813-4a18-9b27-351572986bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:27.737060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65affc74-5813-4a18-9b27-351572986bb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:27.737767 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65affc74-5813-4a18-9b27-351572986bb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:27.739692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65affc74-5813-4a18-9b27-351572986bb9 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:27.779496 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:27.781138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c6b905dc-2742-4747-a377-2c59f12a9f7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef08f54dbf9a4f7f962361d580aaa8bc'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:27.796690 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c6b905dc-2742-4747-a377-2c59f12a9f7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef08f54dbf9a4f7f962361d580aaa8bc'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:27.869873 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 869/1717] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:27 2022] POST /identity/v3/projects => generated 365 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:27.884084 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65affc74-5813-4a18-9b27-351572986bb9 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:27.897242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c6b905dc-2742-4747-a377-2c59f12a9f7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef08f54dbf9a4f7f962361d580aaa8bc'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:27.897876 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c6b905dc-2742-4747-a377-2c59f12a9f7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef08f54dbf9a4f7f962361d580aaa8bc'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:27.899888 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c6b905dc-2742-4747-a377-2c59f12a9f7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef08f54dbf9a4f7f962361d580aaa8bc'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:27.938933 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 849/1718] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:50:27 2022] DELETE /identity/v3/users/ef08f54dbf9a4f7f962361d580aaa8bc => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:50:27.948661 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0a2f8-ecc7-492a-a7d1-9220eee2dc77 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:27.949089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0a2f8-ecc7-492a-a7d1-9220eee2dc77 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:27.949555 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f0a2f8-ecc7-492a-a7d1-9220eee2dc77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:27.995304 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-96f0a2f8-ecc7-492a-a7d1-9220eee2dc77 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:27.996939 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-96f0a2f8-ecc7-492a-a7d1-9220eee2dc77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:28.013469 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65affc74-5813-4a18-9b27-351572986bb9 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:28.127886 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2893ad-dbd2-44b2-a11c-9274dc256ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:28.130646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2893ad-dbd2-44b2-a11c-9274dc256ae0 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:28.131052 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2893ad-dbd2-44b2-a11c-9274dc256ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:28.131577 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2893ad-dbd2-44b2-a11c-9274dc256ae0 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:28.132383 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2893ad-dbd2-44b2-a11c-9274dc256ae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:28.136248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2893ad-dbd2-44b2-a11c-9274dc256ae0 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:28.138388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:28.192814 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 850/1719] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:28.208322 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19048822-76fe-4cd3-94f1-d37b39637890 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:28.208625 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19048822-76fe-4cd3-94f1-d37b39637890 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:28.209446 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19048822-76fe-4cd3-94f1-d37b39637890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:28.231697 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-19048822-76fe-4cd3-94f1-d37b39637890 None None] MFA Rules not processed for user `c700b0adb9f54947a88c1baa86accb9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:28.232936 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-19048822-76fe-4cd3-94f1-d37b39637890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:28.272867 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 870/1720] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:27 2022] POST /identity/v3/users => generated 283 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:28.278208 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2893ad-dbd2-44b2-a11c-9274dc256ae0 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:28.282092 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-4b2893ad-dbd2-44b2-a11c-9274dc256ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:28.364512 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 851/1721] 10.222.0.13 () {58 vars in 1009 bytes} [Fri May 27 19:50:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:28.368909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19048822-76fe-4cd3-94f1-d37b39637890 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:28.375665 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-19048822-76fe-4cd3-94f1-d37b39637890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:28.398879 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19048822-76fe-4cd3-94f1-d37b39637890 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:28.404231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6190d-04ba-4b4b-ad32-c728765823a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:28.405366 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6190d-04ba-4b4b-ad32-c728765823a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:28.405861 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6190d-04ba-4b4b-ad32-c728765823a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:28.406364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6190d-04ba-4b4b-ad32-c728765823a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:28.407411 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a6190d-04ba-4b4b-ad32-c728765823a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:28.407588 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2893ad-dbd2-44b2-a11c-9274dc256ae0 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:28.409670 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a6190d-04ba-4b4b-ad32-c728765823a9 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:28.411532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf48f12-1129-4e7a-9066-a5f10b2cd1ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:28.412194 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf48f12-1129-4e7a-9066-a5f10b2cd1ba service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:28.412479 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf48f12-1129-4e7a-9066-a5f10b2cd1ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:28.412767 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf48f12-1129-4e7a-9066-a5f10b2cd1ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:28.423877 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf48f12-1129-4e7a-9066-a5f10b2cd1ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:28.426922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf48f12-1129-4e7a-9066-a5f10b2cd1ba service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:28.444415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 871/1722] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:28.456259 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 852/1723] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:28.464751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf48f12-1129-4e7a-9066-a5f10b2cd1ba service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:28.476303 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf48f12-1129-4e7a-9066-a5f10b2cd1ba service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:28.484974 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f61cbc-da87-4b46-a94e-fcf42906a45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:28.486035 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f61cbc-da87-4b46-a94e-fcf42906a45f admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:28.486547 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f61cbc-da87-4b46-a94e-fcf42906a45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:28.487052 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f61cbc-da87-4b46-a94e-fcf42906a45f admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:28.487953 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f61cbc-da87-4b46-a94e-fcf42906a45f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:28.489823 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f61cbc-da87-4b46-a94e-fcf42906a45f admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:28.510039 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6190d-04ba-4b4b-ad32-c728765823a9 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:28.510822 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 872/1724] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:28 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:28.564311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6190d-04ba-4b4b-ad32-c728765823a9 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:28.575289 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b63b0a-8795-4f70-8fba-ce62caa35e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:28.576936 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b63b0a-8795-4f70-8fba-ce62caa35e64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:28.577771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b63b0a-8795-4f70-8fba-ce62caa35e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:28.578560 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b63b0a-8795-4f70-8fba-ce62caa35e64 admin admin] PATH_INFO: `/v3/projects/4520fe04426a43a3b9fb356247e4fae9/users/ea89b2d27cc04f3b8795bb7c46325d73/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:28.607580 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b63b0a-8795-4f70-8fba-ce62caa35e64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4520fe04426a43a3b9fb356247e4fae9, user_id=ea89b2d27cc04f3b8795bb7c46325d73, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:28.611914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b63b0a-8795-4f70-8fba-ce62caa35e64 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:28.703203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 853/1725] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:28 2022] PUT /identity/v3/projects/4520fe04426a43a3b9fb356247e4fae9/users/ea89b2d27cc04f3b8795bb7c46325d73/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:28.714235 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746fc39-5505-4d82-a87a-0588119cd92f None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:28.714537 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746fc39-5505-4d82-a87a-0588119cd92f None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:28.714872 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6746fc39-5505-4d82-a87a-0588119cd92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:28.758567 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-6746fc39-5505-4d82-a87a-0588119cd92f None None] MFA Rules not processed for user `ea89b2d27cc04f3b8795bb7c46325d73`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:28.760667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-6746fc39-5505-4d82-a87a-0588119cd92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:28.787045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b63b0a-8795-4f70-8fba-ce62caa35e64 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:28.794587 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-30b63b0a-8795-4f70-8fba-ce62caa35e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:28.864972 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 873/1726] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:28 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:28.877178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6746fc39-5505-4d82-a87a-0588119cd92f None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:28.880616 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:28.881945 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-6746fc39-5505-4d82-a87a-0588119cd92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:28.927373 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b63b0a-8795-4f70-8fba-ce62caa35e64 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:28.944245 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e8e61-919b-47d0-947e-39acbd94d075 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37bc6d0abb454de19e1ab2f46bf4a593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:28.944754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e8e61-919b-47d0-947e-39acbd94d075 service swift] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:28.945651 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e8e61-919b-47d0-947e-39acbd94d075 service swift] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:28.946007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e8e61-919b-47d0-947e-39acbd94d075 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:28.986374 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:29.012414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6746fc39-5505-4d82-a87a-0588119cd92f None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:29.018561 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956a8f6-b205-493e-9aef-9e7b416b021c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:29.019350 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956a8f6-b205-493e-9aef-9e7b416b021c service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:29.019794 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956a8f6-b205-493e-9aef-9e7b416b021c service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:29.020389 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956a8f6-b205-493e-9aef-9e7b416b021c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:29.045011 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956a8f6-b205-493e-9aef-9e7b416b021c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:29.050224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956a8f6-b205-493e-9aef-9e7b416b021c service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:29.087156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 874/1727] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:28 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:50:29.117068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e8e61-919b-47d0-947e-39acbd94d075 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:29.122126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e8e61-919b-47d0-947e-39acbd94d075 service swift] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:29.144457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 854/1728] 10.222.0.13 () {60 vars in 1384 bytes} [Fri May 27 19:50:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 19:50:29.223741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff908bde-28c4-4f89-86ff-9245d4aa2364 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:29.224097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff908bde-28c4-4f89-86ff-9245d4aa2364 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:29.224575 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff908bde-28c4-4f89-86ff-9245d4aa2364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:29.275135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-ff908bde-28c4-4f89-86ff-9245d4aa2364 None None] MFA Rules not processed for user `84c7224a6592470a9c873804361d5538`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:29.280402 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ff908bde-28c4-4f89-86ff-9245d4aa2364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:29.397371 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 875/1729] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:29 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:30.084845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebcac9-ecc0-4087-b5eb-3ebbe9132b8c None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:30.087949 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:30.088877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebcac9-ecc0-4087-b5eb-3ebbe9132b8c None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:30.089358 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebcac9-ecc0-4087-b5eb-3ebbe9132b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:30.176344 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-31ebcac9-ecc0-4087-b5eb-3ebbe9132b8c None None] MFA Rules not processed for user `ed8754ddcfd54c24ac923db0fedc1606`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:30.178117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-31ebcac9-ecc0-4087-b5eb-3ebbe9132b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:30.236479 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:30.463404 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 855/1730] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:30.584147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff908bde-28c4-4f89-86ff-9245d4aa2364 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:30.588000 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ff908bde-28c4-4f89-86ff-9245d4aa2364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:30.791812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff908bde-28c4-4f89-86ff-9245d4aa2364 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:30.791812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937add7a-84c0-4a57-862e-9584c4bbcf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:30.791812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937add7a-84c0-4a57-862e-9584c4bbcf95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:30.791812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937add7a-84c0-4a57-862e-9584c4bbcf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:30.793420 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937add7a-84c0-4a57-862e-9584c4bbcf95 admin admin] PATH_INFO: `/v3/projects/bffe93747b8745d587fdc5a26781d993` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:30.793420 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937add7a-84c0-4a57-862e-9584c4bbcf95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bffe93747b8745d587fdc5a26781d993)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:30.793420 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937add7a-84c0-4a57-862e-9584c4bbcf95 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:31.282413 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-937add7a-84c0-4a57-862e-9584c4bbcf95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bffe93747b8745d587fdc5a26781d993', 'request_id': 'req-937add7a-84c0-4a57-862e-9584c4bbcf95'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:31.290795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-937add7a-84c0-4a57-862e-9584c4bbcf95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bffe93747b8745d587fdc5a26781d993', 'request_id': 'req-937add7a-84c0-4a57-862e-9584c4bbcf95'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:31.305158 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-937add7a-84c0-4a57-862e-9584c4bbcf95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bffe93747b8745d587fdc5a26781d993', 'request_id': 'req-937add7a-84c0-4a57-862e-9584c4bbcf95'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:31.422938 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-937add7a-84c0-4a57-862e-9584c4bbcf95 admin admin] The token cache is being invalidate because project bffe93747b8745d587fdc5a26781d993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:50:31.422938 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-937add7a-84c0-4a57-862e-9584c4bbcf95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:31.422938 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-937add7a-84c0-4a57-862e-9584c4bbcf95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:31.422938 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 876/1731] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:50:30 2022] DELETE /identity/v3/projects/bffe93747b8745d587fdc5a26781d993 => generated 0 bytes in 835 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:31.423233 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ebcac9-ecc0-4087-b5eb-3ebbe9132b8c None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:31.445282 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ebcac9-ecc0-4087-b5eb-3ebbe9132b8c None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:31.538127 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafdfe6-3084-4511-a396-1732519f3f10 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:31.538538 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafdfe6-3084-4511-a396-1732519f3f10 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:31.540929 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eafdfe6-3084-4511-a396-1732519f3f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:31.587643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-6eafdfe6-3084-4511-a396-1732519f3f10 None None] MFA Rules not processed for user `3c9aa6527c13466bbe5dce68a3b81ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:31.594799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-6eafdfe6-3084-4511-a396-1732519f3f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:31.633613 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a83ee-f176-4def-82cb-44011f29bdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:31.634807 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a83ee-f176-4def-82cb-44011f29bdab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:31.635353 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a83ee-f176-4def-82cb-44011f29bdab admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:31.635570 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a83ee-f176-4def-82cb-44011f29bdab admin admin] PATH_INFO: `/v3/users/e491f573c643449da845954bc5b659a8` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:31.638158 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a83ee-f176-4def-82cb-44011f29bdab admin admin] RBAC: Authorizing `identity:delete_user(user_id=e491f573c643449da845954bc5b659a8)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:31.640741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a83ee-f176-4def-82cb-44011f29bdab admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:31.762976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 877/1732] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:31.762976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d4640-0506-4a37-a363-8f8910b0f892 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:31.762976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d4640-0506-4a37-a363-8f8910b0f892 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:31.762976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d4640-0506-4a37-a363-8f8910b0f892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:31.800651 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-309d4640-0506-4a37-a363-8f8910b0f892 None None] MFA Rules not processed for user `dd34f15400c4476bbb2161dbe1b8b2b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:31.800651 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-309d4640-0506-4a37-a363-8f8910b0f892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:31.862791 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:32.024639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 878/1733] 10.222.0.17 () {58 vars in 1015 bytes} [Fri May 27 19:50:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:32.027375 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ef0c3-b72e-4cd0-b787-03f00c60a38b None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:32.027680 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ef0c3-b72e-4cd0-b787-03f00c60a38b None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:32.027970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ef0c3-b72e-4cd0-b787-03f00c60a38b None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:32.029646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 879/1734] 10.222.0.13 () {54 vars in 921 bytes} [Fri May 27 19:50:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:50:32.030667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d84b85-8710-4618-a544-37ddf68ed90f None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:32.030957 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d84b85-8710-4618-a544-37ddf68ed90f None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:32.046479 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d84b85-8710-4618-a544-37ddf68ed90f None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:32.058507 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 880/1735] 10.222.0.13 () {54 vars in 912 bytes} [Fri May 27 19:50:32 2022] GET /identity => generated 270 bytes in 28 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:50:32.064597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17c250-e5be-46eb-8669-e308edf7de67 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:32.064893 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17c250-e5be-46eb-8669-e308edf7de67 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:32.065305 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17c250-e5be-46eb-8669-e308edf7de67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:32.102786 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-5b3a83ee-f176-4def-82cb-44011f29bdab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e491f573c643449da845954bc5b659a8'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:32.161384 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-5b3a83ee-f176-4def-82cb-44011f29bdab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e491f573c643449da845954bc5b659a8'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:32.171507 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-5a17c250-e5be-46eb-8669-e308edf7de67 None None] MFA Rules not processed for user `ffc2ed13a0c84f128b795002892efc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:32.198050 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-5a17c250-e5be-46eb-8669-e308edf7de67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:32.268399 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-5b3a83ee-f176-4def-82cb-44011f29bdab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e491f573c643449da845954bc5b659a8'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:32.269071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-5b3a83ee-f176-4def-82cb-44011f29bdab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e491f573c643449da845954bc5b659a8'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:32.269712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-5b3a83ee-f176-4def-82cb-44011f29bdab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e491f573c643449da845954bc5b659a8'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:32.310121 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 856/1736] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:50:31 2022] DELETE /identity/v3/users/e491f573c643449da845954bc5b659a8 => generated 0 bytes in 888 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:32.317953 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ac8d0-af23-4b6c-bcc7-a2a1a1f2209a None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:32.318257 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ac8d0-af23-4b6c-bcc7-a2a1a1f2209a None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:32.318589 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ac8d0-af23-4b6c-bcc7-a2a1a1f2209a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:32.354657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 881/1737] 10.222.0.13 () {58 vars in 1018 bytes} [Fri May 27 19:50:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:50:32.358084 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17c250-e5be-46eb-8669-e308edf7de67 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:32.361024 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-5a17c250-e5be-46eb-8669-e308edf7de67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:32.391932 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17c250-e5be-46eb-8669-e308edf7de67 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:32.397628 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-c60ac8d0-af23-4b6c-bcc7-a2a1a1f2209a None None] MFA Rules not processed for user `dd34f15400c4476bbb2161dbe1b8b2b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:32.400838 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-c60ac8d0-af23-4b6c-bcc7-a2a1a1f2209a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:32.546275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa258bcb-580a-4cda-88d5-9aa7a59e6ede service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:32.554304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa258bcb-580a-4cda-88d5-9aa7a59e6ede service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:32.555198 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa258bcb-580a-4cda-88d5-9aa7a59e6ede service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:32.555843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa258bcb-580a-4cda-88d5-9aa7a59e6ede service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:32.582222 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-fa258bcb-580a-4cda-88d5-9aa7a59e6ede service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:32.610441 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 857/1738] 10.222.0.13 () {58 vars in 1009 bytes} [Fri May 27 19:50:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:32.619502 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ac8d0-af23-4b6c-bcc7-a2a1a1f2209a None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:32.619996 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa258bcb-580a-4cda-88d5-9aa7a59e6ede service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:32.622178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-c60ac8d0-af23-4b6c-bcc7-a2a1a1f2209a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:32.625655 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa258bcb-580a-4cda-88d5-9aa7a59e6ede service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:32.656102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 882/1739] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:32.656102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa258bcb-580a-4cda-88d5-9aa7a59e6ede service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:32.677234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa258bcb-580a-4cda-88d5-9aa7a59e6ede service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:32.682534 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4431f-47d0-4ddb-a65a-e9fe000b91a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:32.684713 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4431f-47d0-4ddb-a65a-e9fe000b91a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:32.684713 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4431f-47d0-4ddb-a65a-e9fe000b91a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:32.684713 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce4431f-47d0-4ddb-a65a-e9fe000b91a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:32.712697 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce4431f-47d0-4ddb-a65a-e9fe000b91a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:32.726395 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce4431f-47d0-4ddb-a65a-e9fe000b91a2 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:32.777448 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 883/1740] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:32.819469 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ac8d0-af23-4b6c-bcc7-a2a1a1f2209a None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:32.827178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82e99f-120d-4a85-89e5-421d116dcfe8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2ed13a0c84f128b795002892efc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:32.827995 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82e99f-120d-4a85-89e5-421d116dcfe8 service placement] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:32.828387 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82e99f-120d-4a85-89e5-421d116dcfe8 service placement] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:32.828999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82e99f-120d-4a85-89e5-421d116dcfe8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:32.845881 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82e99f-120d-4a85-89e5-421d116dcfe8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:32.852367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82e99f-120d-4a85-89e5-421d116dcfe8 service placement] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:32.874219 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 858/1741] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:33.215520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:34.163954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4431f-47d0-4ddb-a65a-e9fe000b91a2 service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:34.167572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-bce4431f-47d0-4ddb-a65a-e9fe000b91a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:34.389405 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cf278-be90-4ccb-b332-ffe7ff24dec9 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:34.389405 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cf278-be90-4ccb-b332-ffe7ff24dec9 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:34.389405 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472cf278-be90-4ccb-b332-ffe7ff24dec9 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:34.391359 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 859/1742] 10.222.0.13 () {54 vars in 921 bytes} [Fri May 27 19:50:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:50:34.489057 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22995ea4-22fb-49e7-936f-0e3dd8b89ceb None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:34.489835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22995ea4-22fb-49e7-936f-0e3dd8b89ceb None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:34.490395 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22995ea4-22fb-49e7-936f-0e3dd8b89ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:34.606154 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-22995ea4-22fb-49e7-936f-0e3dd8b89ceb None None] MFA Rules not processed for user `ffc2ed13a0c84f128b795002892efc2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:34.607413 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-22995ea4-22fb-49e7-936f-0e3dd8b89ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:34.618802 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce4431f-47d0-4ddb-a65a-e9fe000b91a2 service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:34.647975 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735fbe9-541d-4fdb-8c31-ab4234280dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:34.649105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735fbe9-541d-4fdb-8c31-ab4234280dba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:34.649399 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735fbe9-541d-4fdb-8c31-ab4234280dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:34.649681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735fbe9-541d-4fdb-8c31-ab4234280dba admin admin] PATH_INFO: `/v3/projects/5e3d495d42d8454391c8a81d1383d74c` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:34.651156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735fbe9-541d-4fdb-8c31-ab4234280dba admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e3d495d42d8454391c8a81d1383d74c)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:34.654573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735fbe9-541d-4fdb-8c31-ab4234280dba admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:34.660456 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 860/1743] 10.222.0.13 () {58 vars in 1018 bytes} [Fri May 27 19:50:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:50:34.670342 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22995ea4-22fb-49e7-936f-0e3dd8b89ceb None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:34.676831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-22995ea4-22fb-49e7-936f-0e3dd8b89ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:34.726987 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:34.738360 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22995ea4-22fb-49e7-936f-0e3dd8b89ceb None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:34.758788 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76d4e0-0291-4e1c-a86a-2ad7a9ce0f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:34.760946 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76d4e0-0291-4e1c-a86a-2ad7a9ce0f83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:34.764667 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76d4e0-0291-4e1c-a86a-2ad7a9ce0f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:34.765074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76d4e0-0291-4e1c-a86a-2ad7a9ce0f83 admin admin] PATH_INFO: `/v3/users/c511d932107a41daab469b58d4ee9d60` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:34.766507 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76d4e0-0291-4e1c-a86a-2ad7a9ce0f83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c511d932107a41daab469b58d4ee9d60)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:34.779833 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76d4e0-0291-4e1c-a86a-2ad7a9ce0f83 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:34.905775 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c735fbe9-541d-4fdb-8c31-ab4234280dba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e3d495d42d8454391c8a81d1383d74c', 'request_id': 'req-c735fbe9-541d-4fdb-8c31-ab4234280dba'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:34.916350 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c735fbe9-541d-4fdb-8c31-ab4234280dba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e3d495d42d8454391c8a81d1383d74c', 'request_id': 'req-c735fbe9-541d-4fdb-8c31-ab4234280dba'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:34.923510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c735fbe9-541d-4fdb-8c31-ab4234280dba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e3d495d42d8454391c8a81d1383d74c', 'request_id': 'req-c735fbe9-541d-4fdb-8c31-ab4234280dba'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:35.045757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c735fbe9-541d-4fdb-8c31-ab4234280dba admin admin] The token cache is being invalidate because project 5e3d495d42d8454391c8a81d1383d74c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:50:35.048435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c735fbe9-541d-4fdb-8c31-ab4234280dba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:35.051678 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c735fbe9-541d-4fdb-8c31-ab4234280dba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:35.066637 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 884/1744] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:50:34 2022] DELETE /identity/v3/projects/5e3d495d42d8454391c8a81d1383d74c => generated 0 bytes in 899 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:35.067807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735fbe9-541d-4fdb-8c31-ab4234280dba admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:35.075377 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-c735fbe9-541d-4fdb-8c31-ab4234280dba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:35.126513 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-4d76d4e0-0291-4e1c-a86a-2ad7a9ce0f83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c511d932107a41daab469b58d4ee9d60'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:35.155427 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-4d76d4e0-0291-4e1c-a86a-2ad7a9ce0f83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c511d932107a41daab469b58d4ee9d60'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:35.243872 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735fbe9-541d-4fdb-8c31-ab4234280dba admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:35.250889 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e13873-03ba-4bdf-af57-c7908842afbc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2ed13a0c84f128b795002892efc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:35.251879 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e13873-03ba-4bdf-af57-c7908842afbc service placement] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:35.252378 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e13873-03ba-4bdf-af57-c7908842afbc service placement] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:35.252900 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e13873-03ba-4bdf-af57-c7908842afbc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:35.255195 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-e6e13873-03ba-4bdf-af57-c7908842afbc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:35.264035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:35.264980 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-4d76d4e0-0291-4e1c-a86a-2ad7a9ce0f83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c511d932107a41daab469b58d4ee9d60'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:35.265852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-4d76d4e0-0291-4e1c-a86a-2ad7a9ce0f83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c511d932107a41daab469b58d4ee9d60'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:35.267098 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-4d76d4e0-0291-4e1c-a86a-2ad7a9ce0f83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c511d932107a41daab469b58d4ee9d60'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:35.271381 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e13873-03ba-4bdf-af57-c7908842afbc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:35.274240 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e13873-03ba-4bdf-af57-c7908842afbc service placement] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:35.285359 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 861/1745] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:50:34 2022] DELETE /identity/v3/users/c511d932107a41daab469b58d4ee9d60 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:35.286928 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315b54d-f054-4502-be1c-6c1282bc3e36 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:35.287289 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315b54d-f054-4502-be1c-6c1282bc3e36 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:35.287648 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4315b54d-f054-4502-be1c-6c1282bc3e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:35.303644 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 885/1746] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:35.327486 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-4315b54d-f054-4502-be1c-6c1282bc3e36 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:35.331607 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-4315b54d-f054-4502-be1c-6c1282bc3e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:35.428343 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 862/1747] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:35.431027 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c494d-5963-425c-93ce-699888e6f679 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:35.431622 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c494d-5963-425c-93ce-699888e6f679 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:35.432168 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c494d-5963-425c-93ce-699888e6f679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:35.485851 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-d70c494d-5963-425c-93ce-699888e6f679 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:35.487749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d70c494d-5963-425c-93ce-699888e6f679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:35.504487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38464d49-e1c2-4688-ab4e-c90de8663942 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:35.505125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38464d49-e1c2-4688-ab4e-c90de8663942 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:35.505574 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38464d49-e1c2-4688-ab4e-c90de8663942 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:35.508419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 863/1748] 10.222.0.13 () {54 vars in 921 bytes} [Fri May 27 19:50:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:50:35.536011 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 886/1749] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:35.536198 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38464d49-e1c2-4688-ab4e-c90de8663942 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:35.541982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4767c-49e0-405f-b0cc-187eea8bf88a None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:35.542277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4767c-49e0-405f-b0cc-187eea8bf88a None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:35.542558 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4767c-49e0-405f-b0cc-187eea8bf88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:35.578767 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-16f4767c-49e0-405f-b0cc-187eea8bf88a None None] MFA Rules not processed for user `7542e1fd160e4b2c9d25accf19aad19f`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:35.579347 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-16f4767c-49e0-405f-b0cc-187eea8bf88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:35.584367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38464d49-e1c2-4688-ab4e-c90de8663942 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:35.607735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b0807-583d-4620-92c9-d9f62486b2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:35.608491 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b0807-583d-4620-92c9-d9f62486b2df admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:35.608822 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b0807-583d-4620-92c9-d9f62486b2df admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:35.609091 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25b0807-583d-4620-92c9-d9f62486b2df admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:35.609840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b0807-583d-4620-92c9-d9f62486b2df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:50:35.610154 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b0807-583d-4620-92c9-d9f62486b2df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:35.634992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25b0807-583d-4620-92c9-d9f62486b2df admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:35.672773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 864/1750] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:50:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:50:35.677786 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b0807-583d-4620-92c9-d9f62486b2df admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:35.697533 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25b0807-583d-4620-92c9-d9f62486b2df admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:35.705193 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad46b7-b582-45de-b977-2753967af4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:35.706103 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad46b7-b582-45de-b977-2753967af4c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:35.709082 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad46b7-b582-45de-b977-2753967af4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:35.709082 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ad46b7-b582-45de-b977-2753967af4c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:35.711816 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad46b7-b582-45de-b977-2753967af4c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:35.714653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ad46b7-b582-45de-b977-2753967af4c4 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:35.715877 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 887/1751] 10.222.0.13 () {58 vars in 1018 bytes} [Fri May 27 19:50:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:50:35.751795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4767c-49e0-405f-b0cc-187eea8bf88a None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:35.761593 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-16f4767c-49e0-405f-b0cc-187eea8bf88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:35.794328 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4767c-49e0-405f-b0cc-187eea8bf88a None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:35.805846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08387e95-ed5e-4bc1-8c7b-5d7c472f20b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:35.806548 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08387e95-ed5e-4bc1-8c7b-5d7c472f20b8 service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:35.807025 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08387e95-ed5e-4bc1-8c7b-5d7c472f20b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:35.807328 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08387e95-ed5e-4bc1-8c7b-5d7c472f20b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:35.819149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08387e95-ed5e-4bc1-8c7b-5d7c472f20b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:35.822295 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08387e95-ed5e-4bc1-8c7b-5d7c472f20b8 service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:35.834027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 865/1752] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:35 2022] POST /identity/v3/projects => generated 367 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:35.847351 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 888/1753] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:50:35.850334 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad46b7-b582-45de-b977-2753967af4c4 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:35.977961 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08387e95-ed5e-4bc1-8c7b-5d7c472f20b8 service nova] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:35.979490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-08387e95-ed5e-4bc1-8c7b-5d7c472f20b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:36.002983 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08387e95-ed5e-4bc1-8c7b-5d7c472f20b8 service nova] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:36.019232 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ad46b7-b582-45de-b977-2753967af4c4 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:36.025706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2f3f9-7bd8-4bdc-be20-63079ec8f9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:36.026459 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2f3f9-7bd8-4bdc-be20-63079ec8f9df admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:36.026753 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2f3f9-7bd8-4bdc-be20-63079ec8f9df admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:36.027081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2f3f9-7bd8-4bdc-be20-63079ec8f9df admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:36.028093 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2f3f9-7bd8-4bdc-be20-63079ec8f9df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:36.040941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2f3f9-7bd8-4bdc-be20-63079ec8f9df admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:36.152650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7079e-6e7a-434a-a5e5-a77d20964544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:36.160326 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7079e-6e7a-434a-a5e5-a77d20964544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:36.160326 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7079e-6e7a-434a-a5e5-a77d20964544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:36.160326 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7079e-6e7a-434a-a5e5-a77d20964544 admin admin] PATH_INFO: `/v3/projects/7c727f840dd646639798a916ff118e33` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:36.160326 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7079e-6e7a-434a-a5e5-a77d20964544 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c727f840dd646639798a916ff118e33)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:36.160326 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7079e-6e7a-434a-a5e5-a77d20964544 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:36.238738 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 866/1754] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:35 2022] POST /identity/v3/users => generated 284 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:36.239898 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2f3f9-7bd8-4bdc-be20-63079ec8f9df admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:36.261315 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2f3f9-7bd8-4bdc-be20-63079ec8f9df admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:36.270662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ce9fb-bf08-46ce-b6e1-10df3ee25ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:36.271352 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ce9fb-bf08-46ce-b6e1-10df3ee25ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:36.271638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ce9fb-bf08-46ce-b6e1-10df3ee25ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:36.272054 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34ce9fb-bf08-46ce-b6e1-10df3ee25ca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:36.272741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ce9fb-bf08-46ce-b6e1-10df3ee25ca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:36.274477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34ce9fb-bf08-46ce-b6e1-10df3ee25ca0 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:36.285565 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 867/1755] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:50:36.290041 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ce9fb-bf08-46ce-b6e1-10df3ee25ca0 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:36.300937 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34ce9fb-bf08-46ce-b6e1-10df3ee25ca0 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:36.306445 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726bbb5-e6af-4787-a958-5d6440ca8557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:36.307296 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726bbb5-e6af-4787-a958-5d6440ca8557 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:36.307584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726bbb5-e6af-4787-a958-5d6440ca8557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:36.307863 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726bbb5-e6af-4787-a958-5d6440ca8557 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:36.308648 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e726bbb5-e6af-4787-a958-5d6440ca8557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:36.310346 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e726bbb5-e6af-4787-a958-5d6440ca8557 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:36.322839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 868/1756] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:36.327322 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726bbb5-e6af-4787-a958-5d6440ca8557 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:36.339070 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726bbb5-e6af-4787-a958-5d6440ca8557 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:36.350660 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-17a7079e-6e7a-434a-a5e5-a77d20964544 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c727f840dd646639798a916ff118e33', 'request_id': 'req-17a7079e-6e7a-434a-a5e5-a77d20964544'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:36.364022 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-17a7079e-6e7a-434a-a5e5-a77d20964544 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c727f840dd646639798a916ff118e33', 'request_id': 'req-17a7079e-6e7a-434a-a5e5-a77d20964544'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:36.371652 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-17a7079e-6e7a-434a-a5e5-a77d20964544 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c727f840dd646639798a916ff118e33', 'request_id': 'req-17a7079e-6e7a-434a-a5e5-a77d20964544'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:36.404565 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8427c0bd-79ca-4884-9301-94a5a0964f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:36.405412 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427c0bd-79ca-4884-9301-94a5a0964f9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:36.405699 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427c0bd-79ca-4884-9301-94a5a0964f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:36.405980 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8427c0bd-79ca-4884-9301-94a5a0964f9c admin admin] PATH_INFO: `/v3/projects/66dbc9fb07c14093b1d0359143c64574/users/80077d6e856845e98974f61a8e831968/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:36.420491 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8427c0bd-79ca-4884-9301-94a5a0964f9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=66dbc9fb07c14093b1d0359143c64574, user_id=80077d6e856845e98974f61a8e831968, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:36.423304 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8427c0bd-79ca-4884-9301-94a5a0964f9c admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:36.466359 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-17a7079e-6e7a-434a-a5e5-a77d20964544 admin admin] The token cache is being invalidate because project 7c727f840dd646639798a916ff118e33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:50:36.466649 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-17a7079e-6e7a-434a-a5e5-a77d20964544 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:36.467280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-17a7079e-6e7a-434a-a5e5-a77d20964544 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:36.469151 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 889/1757] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:50:35 2022] DELETE /identity/v3/projects/7c727f840dd646639798a916ff118e33 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:50:36.473115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7079e-6e7a-434a-a5e5-a77d20964544 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:36.474440 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-17a7079e-6e7a-434a-a5e5-a77d20964544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:36.486981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7079e-6e7a-434a-a5e5-a77d20964544 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:36.493323 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc871f-5e94-447b-937f-e863b8487d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:36.511331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc871f-5e94-447b-937f-e863b8487d08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:36.511331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc871f-5e94-447b-937f-e863b8487d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:36.511331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc871f-5e94-447b-937f-e863b8487d08 admin admin] PATH_INFO: `/v3/users/84c7224a6592470a9c873804361d5538` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:36.511331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc871f-5e94-447b-937f-e863b8487d08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84c7224a6592470a9c873804361d5538)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:36.511840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 869/1758] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:36 2022] PUT /identity/v3/projects/66dbc9fb07c14093b1d0359143c64574/users/80077d6e856845e98974f61a8e831968/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:50:36.512426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdc871f-5e94-447b-937f-e863b8487d08 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:36.513437 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d898ba4-e829-42b0-905d-3e93b2c42eed None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:36.514075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d898ba4-e829-42b0-905d-3e93b2c42eed None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:36.514360 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d898ba4-e829-42b0-905d-3e93b2c42eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:36.542147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-4d898ba4-e829-42b0-905d-3e93b2c42eed None None] MFA Rules not processed for user `80077d6e856845e98974f61a8e831968`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:36.543338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-4d898ba4-e829-42b0-905d-3e93b2c42eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:36.609248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 870/1759] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:36 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:36.613736 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d898ba4-e829-42b0-905d-3e93b2c42eed None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:36.615136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-4d898ba4-e829-42b0-905d-3e93b2c42eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:36.681193 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d898ba4-e829-42b0-905d-3e93b2c42eed None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:36.685812 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28958566-a963-484d-92d0-10ba991fb0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:36.686474 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28958566-a963-484d-92d0-10ba991fb0de admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:36.686756 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28958566-a963-484d-92d0-10ba991fb0de admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:36.687031 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28958566-a963-484d-92d0-10ba991fb0de admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:36.687737 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28958566-a963-484d-92d0-10ba991fb0de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:36.689751 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28958566-a963-484d-92d0-10ba991fb0de admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:36.715789 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-7bdc871f-5e94-447b-937f-e863b8487d08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84c7224a6592470a9c873804361d5538'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:36.730977 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-7bdc871f-5e94-447b-937f-e863b8487d08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84c7224a6592470a9c873804361d5538'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:36.812405 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 871/1760] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:36 2022] POST /identity/v3/projects => generated 365 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:50:36.816276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28958566-a963-484d-92d0-10ba991fb0de admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:36.862423 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-7bdc871f-5e94-447b-937f-e863b8487d08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84c7224a6592470a9c873804361d5538'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:36.863156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-7bdc871f-5e94-447b-937f-e863b8487d08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84c7224a6592470a9c873804361d5538'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:36.871357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-7bdc871f-5e94-447b-937f-e863b8487d08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84c7224a6592470a9c873804361d5538'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:36.877708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 890/1761] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:50:36 2022] DELETE /identity/v3/users/84c7224a6592470a9c873804361d5538 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:50:36.879263 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28958566-a963-484d-92d0-10ba991fb0de admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:36.943689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242163cd-fe95-4748-b90a-dcc63e1e4432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:36.944366 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242163cd-fe95-4748-b90a-dcc63e1e4432 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:36.945362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242163cd-fe95-4748-b90a-dcc63e1e4432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:36.945655 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242163cd-fe95-4748-b90a-dcc63e1e4432 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:36.946345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242163cd-fe95-4748-b90a-dcc63e1e4432 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:36.948600 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242163cd-fe95-4748-b90a-dcc63e1e4432 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:36.965781 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:37.008591 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 872/1762] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:36 2022] POST /identity/v3/users => generated 289 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:50:37.012475 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc871f-5e94-447b-937f-e863b8487d08 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:37.013926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-7bdc871f-5e94-447b-937f-e863b8487d08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:37.028177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc871f-5e94-447b-937f-e863b8487d08 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:37.033516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201add6e-7acf-447a-807e-3862de8a29c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:37.033762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201add6e-7acf-447a-807e-3862de8a29c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:37.034059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201add6e-7acf-447a-807e-3862de8a29c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:37.034344 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201add6e-7acf-447a-807e-3862de8a29c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:37.035073 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201add6e-7acf-447a-807e-3862de8a29c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:37.036687 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201add6e-7acf-447a-807e-3862de8a29c0 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:37.047512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 891/1763] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:50:37.051568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242163cd-fe95-4748-b90a-dcc63e1e4432 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:37.062169 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242163cd-fe95-4748-b90a-dcc63e1e4432 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:37.067140 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788670c3-844b-4ecb-87c5-c6a03ae460f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:37.068148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788670c3-844b-4ecb-87c5-c6a03ae460f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:37.068442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788670c3-844b-4ecb-87c5-c6a03ae460f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:37.068729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788670c3-844b-4ecb-87c5-c6a03ae460f3 admin admin] PATH_INFO: `/v3/projects/07db11275ba743d4bdb6de057695dd22/users/583e16a526a74c02b3a26736114c9449/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:37.083955 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788670c3-844b-4ecb-87c5-c6a03ae460f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07db11275ba743d4bdb6de057695dd22, user_id=583e16a526a74c02b3a26736114c9449, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:37.086945 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788670c3-844b-4ecb-87c5-c6a03ae460f3 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:37.218751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201add6e-7acf-447a-807e-3862de8a29c0 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:37.220887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-201add6e-7acf-447a-807e-3862de8a29c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:37.221796 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 873/1764] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:37 2022] PUT /identity/v3/projects/07db11275ba743d4bdb6de057695dd22/users/583e16a526a74c02b3a26736114c9449/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:50:37.224348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788670c3-844b-4ecb-87c5-c6a03ae460f3 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:37.237459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201add6e-7acf-447a-807e-3862de8a29c0 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:37.307196 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e97078-9539-488f-8bf0-8eacdece481f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:37.307961 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788670c3-844b-4ecb-87c5-c6a03ae460f3 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:37.308076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e97078-9539-488f-8bf0-8eacdece481f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:37.308376 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e97078-9539-488f-8bf0-8eacdece481f admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:37.308723 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e97078-9539-488f-8bf0-8eacdece481f admin admin] PATH_INFO: `/v3/projects/c95c573fbebf47c192b176d307c2b01e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:37.309983 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e97078-9539-488f-8bf0-8eacdece481f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c95c573fbebf47c192b176d307c2b01e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:37.312398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e97078-9539-488f-8bf0-8eacdece481f admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:37.312908 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecfb8d-7f45-47ef-8bd7-432ab2f32cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:37.313617 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecfb8d-7f45-47ef-8bd7-432ab2f32cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:37.313902 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecfb8d-7f45-47ef-8bd7-432ab2f32cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:37.314182 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecfb8d-7f45-47ef-8bd7-432ab2f32cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:37.314914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecfb8d-7f45-47ef-8bd7-432ab2f32cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:37.316508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecfb8d-7f45-47ef-8bd7-432ab2f32cd4 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:37.329941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 874/1765] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:37 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:37.333919 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecfb8d-7f45-47ef-8bd7-432ab2f32cd4 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:37.345035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecfb8d-7f45-47ef-8bd7-432ab2f32cd4 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:37.349888 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3093f1-c47f-4f24-8146-46c073e5b2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:37.350795 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3093f1-c47f-4f24-8146-46c073e5b2db admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:37.351282 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3093f1-c47f-4f24-8146-46c073e5b2db admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:37.351561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3093f1-c47f-4f24-8146-46c073e5b2db admin admin] PATH_INFO: `/v3/projects/07db11275ba743d4bdb6de057695dd22/users/583e16a526a74c02b3a26736114c9449/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:37.353787 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3093f1-c47f-4f24-8146-46c073e5b2db admin admin] RBAC: Authorizing `identity:create_grant(project_id=07db11275ba743d4bdb6de057695dd22, user_id=583e16a526a74c02b3a26736114c9449, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:37.356451 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3093f1-c47f-4f24-8146-46c073e5b2db admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:37.368986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 875/1766] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:37 2022] PUT /identity/v3/projects/07db11275ba743d4bdb6de057695dd22/users/583e16a526a74c02b3a26736114c9449/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:50:37.378652 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831caefa-7b72-479f-a465-d18ad9ca5e37 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:37.378946 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831caefa-7b72-479f-a465-d18ad9ca5e37 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:37.379231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831caefa-7b72-479f-a465-d18ad9ca5e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:37.406973 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-831caefa-7b72-479f-a465-d18ad9ca5e37 None None] MFA Rules not processed for user `583e16a526a74c02b3a26736114c9449`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:37.408200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-831caefa-7b72-479f-a465-d18ad9ca5e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:37.461088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-43e97078-9539-488f-8bf0-8eacdece481f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c95c573fbebf47c192b176d307c2b01e', 'request_id': 'req-43e97078-9539-488f-8bf0-8eacdece481f'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:37.466991 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-43e97078-9539-488f-8bf0-8eacdece481f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c95c573fbebf47c192b176d307c2b01e', 'request_id': 'req-43e97078-9539-488f-8bf0-8eacdece481f'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:37.473519 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-43e97078-9539-488f-8bf0-8eacdece481f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c95c573fbebf47c192b176d307c2b01e', 'request_id': 'req-43e97078-9539-488f-8bf0-8eacdece481f'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:37.479280 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 876/1767] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:37 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:50:37.490655 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831caefa-7b72-479f-a465-d18ad9ca5e37 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:37.492517 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-831caefa-7b72-479f-a465-d18ad9ca5e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:37.553360 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831caefa-7b72-479f-a465-d18ad9ca5e37 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:37.558991 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-43e97078-9539-488f-8bf0-8eacdece481f admin admin] The token cache is being invalidate because project c95c573fbebf47c192b176d307c2b01e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:50:37.559365 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-43e97078-9539-488f-8bf0-8eacdece481f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:37.560040 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-43e97078-9539-488f-8bf0-8eacdece481f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:37.566568 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 892/1768] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:50:37 2022] DELETE /identity/v3/projects/c95c573fbebf47c192b176d307c2b01e => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:37.568203 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c73c0-4396-41f2-b7d2-b30cff8dd0a0 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:37.568491 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c73c0-4396-41f2-b7d2-b30cff8dd0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:37.568775 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c73c0-4396-41f2-b7d2-b30cff8dd0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:37.608164 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-be0c73c0-4396-41f2-b7d2-b30cff8dd0a0 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:37.609428 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-be0c73c0-4396-41f2-b7d2-b30cff8dd0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:37.611001 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92a9bc-22e8-48b2-994e-bfb11f48f70a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:37.611917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92a9bc-22e8-48b2-994e-bfb11f48f70a service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:37.612291 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92a9bc-22e8-48b2-994e-bfb11f48f70a service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:37.617296 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92a9bc-22e8-48b2-994e-bfb11f48f70a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:37.620736 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-1b92a9bc-22e8-48b2-994e-bfb11f48f70a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:37.693125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 893/1769] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:37.700565 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c630f-5f51-4a00-bbc3-19c6e00bc515 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:37.700910 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c630f-5f51-4a00-bbc3-19c6e00bc515 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:37.701221 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c630f-5f51-4a00-bbc3-19c6e00bc515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:37.721832 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92a9bc-22e8-48b2-994e-bfb11f48f70a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:37.721832 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92a9bc-22e8-48b2-994e-bfb11f48f70a service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:37.744387 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-b75c630f-5f51-4a00-bbc3-19c6e00bc515 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:37.744584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 877/1770] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:37 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:37.745990 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-b75c630f-5f51-4a00-bbc3-19c6e00bc515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:37.755322 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 894/1771] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:37.761261 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92a9bc-22e8-48b2-994e-bfb11f48f70a service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:37.775078 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92a9bc-22e8-48b2-994e-bfb11f48f70a service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:37.785105 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414ec34-2d15-4ff6-bf81-00f1d61710a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:37.785105 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414ec34-2d15-4ff6-bf81-00f1d61710a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:37.785105 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414ec34-2d15-4ff6-bf81-00f1d61710a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:37.785105 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414ec34-2d15-4ff6-bf81-00f1d61710a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:37.785541 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414ec34-2d15-4ff6-bf81-00f1d61710a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:50:37.789429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:37.789994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414ec34-2d15-4ff6-bf81-00f1d61710a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:37.791891 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414ec34-2d15-4ff6-bf81-00f1d61710a0 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:37.805874 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 878/1772] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:50:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:50:37.810099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c630f-5f51-4a00-bbc3-19c6e00bc515 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:37.820348 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c630f-5f51-4a00-bbc3-19c6e00bc515 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:37.825831 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c8b75-74d1-4b08-9460-c3dc1c7ad715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:37.826583 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c8b75-74d1-4b08-9460-c3dc1c7ad715 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:37.826928 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c8b75-74d1-4b08-9460-c3dc1c7ad715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:37.827453 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071c8b75-74d1-4b08-9460-c3dc1c7ad715 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:37.828221 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c8b75-74d1-4b08-9460-c3dc1c7ad715 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:37.830324 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071c8b75-74d1-4b08-9460-c3dc1c7ad715 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:37.940001 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 895/1773] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:37 2022] POST /identity/v3/projects => generated 381 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:50:37.944180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414ec34-2d15-4ff6-bf81-00f1d61710a0 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:37.955382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414ec34-2d15-4ff6-bf81-00f1d61710a0 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:38.041556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef77d27-51ef-4be6-883d-85d13631ed93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:38.041556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef77d27-51ef-4be6-883d-85d13631ed93 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:38.041556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef77d27-51ef-4be6-883d-85d13631ed93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:38.041556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef77d27-51ef-4be6-883d-85d13631ed93 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:38.041906 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef77d27-51ef-4be6-883d-85d13631ed93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:38.041906 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef77d27-51ef-4be6-883d-85d13631ed93 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:38.050062 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c8b75-74d1-4b08-9460-c3dc1c7ad715 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:38.051882 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-071c8b75-74d1-4b08-9460-c3dc1c7ad715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:38.125330 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071c8b75-74d1-4b08-9460-c3dc1c7ad715 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:38.139973 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447ad2e-895e-4f08-a2f5-2a6329274154 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:38.140281 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 879/1774] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:37 2022] POST /identity/v3/users => generated 291 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:38.141944 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447ad2e-895e-4f08-a2f5-2a6329274154 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:38.142341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447ad2e-895e-4f08-a2f5-2a6329274154 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:38.142848 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447ad2e-895e-4f08-a2f5-2a6329274154 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:38.144762 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef77d27-51ef-4be6-883d-85d13631ed93 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:38.145172 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-f447ad2e-895e-4f08-a2f5-2a6329274154 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:38.159380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef77d27-51ef-4be6-883d-85d13631ed93 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:38.165784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8962c5b0-6db5-4961-b283-b3994ac598ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:38.166686 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962c5b0-6db5-4961-b283-b3994ac598ab admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:38.167313 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962c5b0-6db5-4961-b283-b3994ac598ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:38.167801 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962c5b0-6db5-4961-b283-b3994ac598ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:38.169274 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8962c5b0-6db5-4961-b283-b3994ac598ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:38.171645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8962c5b0-6db5-4961-b283-b3994ac598ab admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:38.184173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 880/1775] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:38.188843 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8962c5b0-6db5-4961-b283-b3994ac598ab admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:38.200450 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8962c5b0-6db5-4961-b283-b3994ac598ab admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:38.206065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9bad55-d62f-424e-9d1c-bb8cd15aa0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:38.206802 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9bad55-d62f-424e-9d1c-bb8cd15aa0e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:38.207094 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9bad55-d62f-424e-9d1c-bb8cd15aa0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:38.207452 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9bad55-d62f-424e-9d1c-bb8cd15aa0e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:38.208220 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9bad55-d62f-424e-9d1c-bb8cd15aa0e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:38.210842 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9bad55-d62f-424e-9d1c-bb8cd15aa0e4 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:38.216809 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:38.226961 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 881/1776] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:50:38.228323 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9bad55-d62f-424e-9d1c-bb8cd15aa0e4 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:38.244643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447ad2e-895e-4f08-a2f5-2a6329274154 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:38.245728 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9bad55-d62f-424e-9d1c-bb8cd15aa0e4 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:38.246490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447ad2e-895e-4f08-a2f5-2a6329274154 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:38.251426 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6b466-7d14-473b-b46f-b6530dbd5061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:38.252476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6b466-7d14-473b-b46f-b6530dbd5061 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:38.252857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6b466-7d14-473b-b46f-b6530dbd5061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:38.253174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6b466-7d14-473b-b46f-b6530dbd5061 admin admin] PATH_INFO: `/v3/projects/8f9b4c00dbf34c4e9d645b504137fb62/users/706174b4b505496686ce0175f6ed8fd4/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:38.258256 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 896/1777] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:38 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:50:38.282930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb6b466-7d14-473b-b46f-b6530dbd5061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f9b4c00dbf34c4e9d645b504137fb62, user_id=706174b4b505496686ce0175f6ed8fd4, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:38.285836 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb6b466-7d14-473b-b46f-b6530dbd5061 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:38.402461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 882/1778] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:38 2022] PUT /identity/v3/projects/8f9b4c00dbf34c4e9d645b504137fb62/users/706174b4b505496686ce0175f6ed8fd4/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:50:38.414675 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7848241-902e-4013-b105-ab29841e500e None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:38.415380 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7848241-902e-4013-b105-ab29841e500e None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:38.416073 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7848241-902e-4013-b105-ab29841e500e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:38.449865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-b7848241-902e-4013-b105-ab29841e500e None None] MFA Rules not processed for user `706174b4b505496686ce0175f6ed8fd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:38.451007 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-b7848241-902e-4013-b105-ab29841e500e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:38.561942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 897/1779] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:38.566435 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6b466-7d14-473b-b46f-b6530dbd5061 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:38.663512 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6b466-7d14-473b-b46f-b6530dbd5061 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:38.668550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69536f-6d89-4950-8642-fb69388e108e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:38.671645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69536f-6d89-4950-8642-fb69388e108e admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:38.671645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69536f-6d89-4950-8642-fb69388e108e admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:38.671645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69536f-6d89-4950-8642-fb69388e108e admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:38.671645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69536f-6d89-4950-8642-fb69388e108e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:38.672620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69536f-6d89-4950-8642-fb69388e108e admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:38.771116 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 883/1780] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:38 2022] POST /identity/v3/projects => generated 381 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:38.772148 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7848241-902e-4013-b105-ab29841e500e None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:38.785976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7848241-902e-4013-b105-ab29841e500e None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:38.855143 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e94354-81bd-4205-847a-6e02deb99a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:38.855861 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e94354-81bd-4205-847a-6e02deb99a01 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:38.856156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e94354-81bd-4205-847a-6e02deb99a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:38.856439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e94354-81bd-4205-847a-6e02deb99a01 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:38.857192 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e94354-81bd-4205-847a-6e02deb99a01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:38.859379 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e94354-81bd-4205-847a-6e02deb99a01 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:38.995250 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 898/1781] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:38 2022] POST /identity/v3/users => generated 297 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:50:39.000113 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69536f-6d89-4950-8642-fb69388e108e admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:39.017640 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69536f-6d89-4950-8642-fb69388e108e admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:39.024618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f18338-ff25-4ca7-be0e-7922b15afab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:39.025491 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f18338-ff25-4ca7-be0e-7922b15afab9 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:39.025848 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f18338-ff25-4ca7-be0e-7922b15afab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:39.026376 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f18338-ff25-4ca7-be0e-7922b15afab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:39.027089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f18338-ff25-4ca7-be0e-7922b15afab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:39.028808 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f18338-ff25-4ca7-be0e-7922b15afab9 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:39.050788 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 884/1782] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:38 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:39.055432 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e94354-81bd-4205-847a-6e02deb99a01 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:39.072484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e94354-81bd-4205-847a-6e02deb99a01 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:39.081185 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c691b-c8f5-4b0c-a37b-11e2f43a2ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:39.090843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c691b-c8f5-4b0c-a37b-11e2f43a2ec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:39.091381 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c691b-c8f5-4b0c-a37b-11e2f43a2ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:39.091870 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c691b-c8f5-4b0c-a37b-11e2f43a2ec0 admin admin] PATH_INFO: `/v3/projects/bce1d5722fee4bf892aa29ad12a0e4f7/users/200347f85f274dd6bb321629a36f289b/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:39.157993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c691b-c8f5-4b0c-a37b-11e2f43a2ec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bce1d5722fee4bf892aa29ad12a0e4f7, user_id=200347f85f274dd6bb321629a36f289b, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:39.177620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c691b-c8f5-4b0c-a37b-11e2f43a2ec0 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:39.276533 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 899/1783] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:39 2022] PUT /identity/v3/projects/bce1d5722fee4bf892aa29ad12a0e4f7/users/200347f85f274dd6bb321629a36f289b/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:50:39.282055 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f18338-ff25-4ca7-be0e-7922b15afab9 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:39.295327 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f18338-ff25-4ca7-be0e-7922b15afab9 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:39.396836 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d10f52-93e4-416e-ab55-ff679f873147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:39.398290 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d10f52-93e4-416e-ab55-ff679f873147 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:39.398972 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d10f52-93e4-416e-ab55-ff679f873147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:39.399967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d10f52-93e4-416e-ab55-ff679f873147 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:39.400623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d10f52-93e4-416e-ab55-ff679f873147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:39.402523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d10f52-93e4-416e-ab55-ff679f873147 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:39.416552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 885/1784] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:39 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:39.421015 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c691b-c8f5-4b0c-a37b-11e2f43a2ec0 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:39.441896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c691b-c8f5-4b0c-a37b-11e2f43a2ec0 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:39.471567 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f317ab1-3e45-4c5e-9b3e-f8285753b7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:39.472752 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f317ab1-3e45-4c5e-9b3e-f8285753b7ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:39.473307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f317ab1-3e45-4c5e-9b3e-f8285753b7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:39.473806 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f317ab1-3e45-4c5e-9b3e-f8285753b7ff admin admin] PATH_INFO: `/v3/projects/bce1d5722fee4bf892aa29ad12a0e4f7/users/200347f85f274dd6bb321629a36f289b/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:39.476667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f317ab1-3e45-4c5e-9b3e-f8285753b7ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=bce1d5722fee4bf892aa29ad12a0e4f7, user_id=200347f85f274dd6bb321629a36f289b, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:39.481085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f317ab1-3e45-4c5e-9b3e-f8285753b7ff admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:39.494541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 900/1785] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:39 2022] PUT /identity/v3/projects/bce1d5722fee4bf892aa29ad12a0e4f7/users/200347f85f274dd6bb321629a36f289b/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:39.497406 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:39.504844 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6346cf7-83e9-45a8-9f48-3638789e2dfa None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:39.511077 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6346cf7-83e9-45a8-9f48-3638789e2dfa None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:39.511077 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6346cf7-83e9-45a8-9f48-3638789e2dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:39.580752 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-c6346cf7-83e9-45a8-9f48-3638789e2dfa None None] MFA Rules not processed for user `200347f85f274dd6bb321629a36f289b`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:39.584817 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-c6346cf7-83e9-45a8-9f48-3638789e2dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:39.681838 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 886/1786] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:39 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:39.838905 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f317ab1-3e45-4c5e-9b3e-f8285753b7ff admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:39.841067 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-8f317ab1-3e45-4c5e-9b3e-f8285753b7ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:39.932230 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f317ab1-3e45-4c5e-9b3e-f8285753b7ff admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:39.936111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec501a9e-98b3-4077-8968-acfb8b6774e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:39.936767 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec501a9e-98b3-4077-8968-acfb8b6774e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:39.946883 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec501a9e-98b3-4077-8968-acfb8b6774e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:39.946883 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec501a9e-98b3-4077-8968-acfb8b6774e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:39.947242 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec501a9e-98b3-4077-8968-acfb8b6774e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:39.948501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec501a9e-98b3-4077-8968-acfb8b6774e6 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:39.958966 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 901/1787] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:40.012173 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:40.139636 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6346cf7-83e9-45a8-9f48-3638789e2dfa None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:40.164023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6346cf7-83e9-45a8-9f48-3638789e2dfa None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:40.167718 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded477ae-e6ce-4d36-9c25-aacbcfe35cbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:40.168471 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded477ae-e6ce-4d36-9c25-aacbcfe35cbe service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:40.168761 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded477ae-e6ce-4d36-9c25-aacbcfe35cbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:40.169077 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded477ae-e6ce-4d36-9c25-aacbcfe35cbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:40.250424 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded477ae-e6ce-4d36-9c25-aacbcfe35cbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:40.252069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded477ae-e6ce-4d36-9c25-aacbcfe35cbe service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:40.261715 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 887/1788] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:40.534262 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:42.327076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:42.718783 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:43.478409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:44.032268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:44.497098 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:44.954530 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:45.576665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:46.828969 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec501a9e-98b3-4077-8968-acfb8b6774e6 service cinder] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:46.830961 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ec501a9e-98b3-4077-8968-acfb8b6774e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:46.922842 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec501a9e-98b3-4077-8968-acfb8b6774e6 service cinder] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:46.927856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f667c-e45a-41a7-881f-9423b52db4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:46.928751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f667c-e45a-41a7-881f-9423b52db4d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:46.929110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f667c-e45a-41a7-881f-9423b52db4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:46.930031 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f667c-e45a-41a7-881f-9423b52db4d2 admin admin] PATH_INFO: `/v3/users/80077d6e856845e98974f61a8e831968` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:46.931638 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f667c-e45a-41a7-881f-9423b52db4d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80077d6e856845e98974f61a8e831968)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:46.933337 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f667c-e45a-41a7-881f-9423b52db4d2 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:47.141828 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f70f667c-e45a-41a7-881f-9423b52db4d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80077d6e856845e98974f61a8e831968'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:47.160094 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f70f667c-e45a-41a7-881f-9423b52db4d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80077d6e856845e98974f61a8e831968'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:47.186704 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded477ae-e6ce-4d36-9c25-aacbcfe35cbe service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:47.203684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded477ae-e6ce-4d36-9c25-aacbcfe35cbe service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:47.213704 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee1435-02a4-48de-9de8-d454a9417d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:47.214729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee1435-02a4-48de-9de8-d454a9417d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:47.224268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee1435-02a4-48de-9de8-d454a9417d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:47.224268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee1435-02a4-48de-9de8-d454a9417d69 admin admin] PATH_INFO: `/v3/users/706174b4b505496686ce0175f6ed8fd4` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:47.224268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee1435-02a4-48de-9de8-d454a9417d69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=706174b4b505496686ce0175f6ed8fd4)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:47.224268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee1435-02a4-48de-9de8-d454a9417d69 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:47.251820 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f70f667c-e45a-41a7-881f-9423b52db4d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80077d6e856845e98974f61a8e831968'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:47.252697 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f70f667c-e45a-41a7-881f-9423b52db4d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80077d6e856845e98974f61a8e831968'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:47.278395 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f70f667c-e45a-41a7-881f-9423b52db4d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80077d6e856845e98974f61a8e831968'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:47.298923 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 902/1789] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:50:46 2022] DELETE /identity/v3/users/80077d6e856845e98974f61a8e831968 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:47.303184 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40419dce-2125-4b6c-8160-acf880c9c98b None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:47.303488 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40419dce-2125-4b6c-8160-acf880c9c98b None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:47.303754 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40419dce-2125-4b6c-8160-acf880c9c98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:47.352501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-40419dce-2125-4b6c-8160-acf880c9c98b None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:47.356337 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-40419dce-2125-4b6c-8160-acf880c9c98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:47.481431 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 903/1790] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:47.503142 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40419dce-2125-4b6c-8160-acf880c9c98b None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:47.505636 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-40419dce-2125-4b6c-8160-acf880c9c98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:47.513757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:47.596855 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40419dce-2125-4b6c-8160-acf880c9c98b None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:47.600755 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751c3ec-09a2-4c80-a7d0-5c8dcf486e16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:47.602058 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751c3ec-09a2-4c80-a7d0-5c8dcf486e16 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:47.602417 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751c3ec-09a2-4c80-a7d0-5c8dcf486e16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:47.602765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751c3ec-09a2-4c80-a7d0-5c8dcf486e16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:47.612665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751c3ec-09a2-4c80-a7d0-5c8dcf486e16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:47.614269 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751c3ec-09a2-4c80-a7d0-5c8dcf486e16 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:47.627387 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 904/1791] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:50:47.740557 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:47.747230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-85ee1435-02a4-48de-9de8-d454a9417d69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '706174b4b505496686ce0175f6ed8fd4'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:47.763036 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-85ee1435-02a4-48de-9de8-d454a9417d69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '706174b4b505496686ce0175f6ed8fd4'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:47.873823 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-85ee1435-02a4-48de-9de8-d454a9417d69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '706174b4b505496686ce0175f6ed8fd4'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:47.874761 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-85ee1435-02a4-48de-9de8-d454a9417d69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '706174b4b505496686ce0175f6ed8fd4'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:47.875693 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-85ee1435-02a4-48de-9de8-d454a9417d69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '706174b4b505496686ce0175f6ed8fd4'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:47.903001 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 888/1792] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:50:47 2022] DELETE /identity/v3/users/706174b4b505496686ce0175f6ed8fd4 => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:47.907575 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321f62e-231d-40ee-979b-d9455e0b1a40 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:47.909807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321f62e-231d-40ee-979b-d9455e0b1a40 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:47.909807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0321f62e-231d-40ee-979b-d9455e0b1a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:47.940174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-0321f62e-231d-40ee-979b-d9455e0b1a40 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:47.945759 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-0321f62e-231d-40ee-979b-d9455e0b1a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:48.028971 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 905/1793] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:50:48.050257 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee1435-02a4-48de-9de8-d454a9417d69 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:48.053490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-85ee1435-02a4-48de-9de8-d454a9417d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:48.167251 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee1435-02a4-48de-9de8-d454a9417d69 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:48.172414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e337d528-a1c2-4b9d-b0ce-1c836766448d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:48.172572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337d528-a1c2-4b9d-b0ce-1c836766448d service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:48.173210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337d528-a1c2-4b9d-b0ce-1c836766448d service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:48.173496 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337d528-a1c2-4b9d-b0ce-1c836766448d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:48.183871 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e337d528-a1c2-4b9d-b0ce-1c836766448d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:48.185732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e337d528-a1c2-4b9d-b0ce-1c836766448d service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:48.199315 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 889/1794] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:48.486770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321f62e-231d-40ee-979b-d9455e0b1a40 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:48.489132 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-0321f62e-231d-40ee-979b-d9455e0b1a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:48.505163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0321f62e-231d-40ee-979b-d9455e0b1a40 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:48.511360 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ec339-b248-43c5-90d9-0d04cb1594e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:48.512504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ec339-b248-43c5-90d9-0d04cb1594e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:48.518881 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ec339-b248-43c5-90d9-0d04cb1594e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:48.519286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ec339-b248-43c5-90d9-0d04cb1594e2 admin admin] PATH_INFO: `/v3/projects/66dbc9fb07c14093b1d0359143c64574` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:48.521770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ec339-b248-43c5-90d9-0d04cb1594e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66dbc9fb07c14093b1d0359143c64574)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:48.524678 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ec339-b248-43c5-90d9-0d04cb1594e2 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:48.654471 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d08ec339-b248-43c5-90d9-0d04cb1594e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66dbc9fb07c14093b1d0359143c64574', 'request_id': 'req-d08ec339-b248-43c5-90d9-0d04cb1594e2'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:48.662548 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d08ec339-b248-43c5-90d9-0d04cb1594e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66dbc9fb07c14093b1d0359143c64574', 'request_id': 'req-d08ec339-b248-43c5-90d9-0d04cb1594e2'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:48.672714 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d08ec339-b248-43c5-90d9-0d04cb1594e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66dbc9fb07c14093b1d0359143c64574', 'request_id': 'req-d08ec339-b248-43c5-90d9-0d04cb1594e2'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:48.757118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d08ec339-b248-43c5-90d9-0d04cb1594e2 admin admin] The token cache is being invalidate because project 66dbc9fb07c14093b1d0359143c64574 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:50:48.757739 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d08ec339-b248-43c5-90d9-0d04cb1594e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:48.758513 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d08ec339-b248-43c5-90d9-0d04cb1594e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:48.760417 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 906/1795] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:50:48 2022] DELETE /identity/v3/projects/66dbc9fb07c14093b1d0359143c64574 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:48.764358 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e337d528-a1c2-4b9d-b0ce-1c836766448d service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:48.777612 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e337d528-a1c2-4b9d-b0ce-1c836766448d service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:48.850598 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf57a0c-f3d6-43cc-aec7-79a7816442b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:48.852111 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf57a0c-f3d6-43cc-aec7-79a7816442b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:48.852738 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf57a0c-f3d6-43cc-aec7-79a7816442b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:48.852857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf57a0c-f3d6-43cc-aec7-79a7816442b6 admin admin] PATH_INFO: `/v3/users/583e16a526a74c02b3a26736114c9449` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:48.854288 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf57a0c-f3d6-43cc-aec7-79a7816442b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=583e16a526a74c02b3a26736114c9449)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:48.857107 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf57a0c-f3d6-43cc-aec7-79a7816442b6 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:49.128992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-cbf57a0c-f3d6-43cc-aec7-79a7816442b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '583e16a526a74c02b3a26736114c9449'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:49.145228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:49.152765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-cbf57a0c-f3d6-43cc-aec7-79a7816442b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '583e16a526a74c02b3a26736114c9449'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:49.237281 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ec339-b248-43c5-90d9-0d04cb1594e2 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:49.239805 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d08ec339-b248-43c5-90d9-0d04cb1594e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:49.249266 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-cbf57a0c-f3d6-43cc-aec7-79a7816442b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '583e16a526a74c02b3a26736114c9449'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:49.250210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-cbf57a0c-f3d6-43cc-aec7-79a7816442b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '583e16a526a74c02b3a26736114c9449'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:49.250843 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-cbf57a0c-f3d6-43cc-aec7-79a7816442b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '583e16a526a74c02b3a26736114c9449'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:49.257745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ec339-b248-43c5-90d9-0d04cb1594e2 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:49.264729 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fafc04-d192-4780-bc2d-e614dd751624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:49.269867 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fafc04-d192-4780-bc2d-e614dd751624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:49.269867 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fafc04-d192-4780-bc2d-e614dd751624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:49.269867 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fafc04-d192-4780-bc2d-e614dd751624 admin admin] PATH_INFO: `/v3/projects/8f9b4c00dbf34c4e9d645b504137fb62` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:49.269867 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fafc04-d192-4780-bc2d-e614dd751624 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f9b4c00dbf34c4e9d645b504137fb62)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:49.277318 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 890/1796] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:50:48 2022] DELETE /identity/v3/users/583e16a526a74c02b3a26736114c9449 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:49.278509 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fafc04-d192-4780-bc2d-e614dd751624 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:49.393307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:49.461661 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf57a0c-f3d6-43cc-aec7-79a7816442b6 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:49.463124 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-cbf57a0c-f3d6-43cc-aec7-79a7816442b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:49.522158 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-e5fafc04-d192-4780-bc2d-e614dd751624 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f9b4c00dbf34c4e9d645b504137fb62', 'request_id': 'req-e5fafc04-d192-4780-bc2d-e614dd751624'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:49.533555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-e5fafc04-d192-4780-bc2d-e614dd751624 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f9b4c00dbf34c4e9d645b504137fb62', 'request_id': 'req-e5fafc04-d192-4780-bc2d-e614dd751624'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:49.542466 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-e5fafc04-d192-4780-bc2d-e614dd751624 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f9b4c00dbf34c4e9d645b504137fb62', 'request_id': 'req-e5fafc04-d192-4780-bc2d-e614dd751624'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:49.580038 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf57a0c-f3d6-43cc-aec7-79a7816442b6 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:49.654725 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae22248-5eaa-418a-8bc8-4dfac16309d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:49.654725 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae22248-5eaa-418a-8bc8-4dfac16309d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:49.654725 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae22248-5eaa-418a-8bc8-4dfac16309d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:49.654725 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae22248-5eaa-418a-8bc8-4dfac16309d1 admin admin] PATH_INFO: `/v3/users/ea89b2d27cc04f3b8795bb7c46325d73` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:49.655133 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae22248-5eaa-418a-8bc8-4dfac16309d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea89b2d27cc04f3b8795bb7c46325d73)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:49.655520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-e5fafc04-d192-4780-bc2d-e614dd751624 admin admin] The token cache is being invalidate because project 8f9b4c00dbf34c4e9d645b504137fb62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:50:49.655622 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-e5fafc04-d192-4780-bc2d-e614dd751624 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:49.658172 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae22248-5eaa-418a-8bc8-4dfac16309d1 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:49.665289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-e5fafc04-d192-4780-bc2d-e614dd751624 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:49.669484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 907/1797] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:50:49 2022] DELETE /identity/v3/projects/8f9b4c00dbf34c4e9d645b504137fb62 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:49.674773 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fafc04-d192-4780-bc2d-e614dd751624 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:49.678673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-e5fafc04-d192-4780-bc2d-e614dd751624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:49.699392 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fafc04-d192-4780-bc2d-e614dd751624 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:49.709381 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb595f37-f28f-416a-8437-b434c92bca67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:49.711134 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb595f37-f28f-416a-8437-b434c92bca67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:49.711545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb595f37-f28f-416a-8437-b434c92bca67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:49.712123 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb595f37-f28f-416a-8437-b434c92bca67 admin admin] PATH_INFO: `/v3/users/200347f85f274dd6bb321629a36f289b` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:49.713837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb595f37-f28f-416a-8437-b434c92bca67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=200347f85f274dd6bb321629a36f289b)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:49.715867 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb595f37-f28f-416a-8437-b434c92bca67 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:49.916827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-bae22248-5eaa-418a-8bc8-4dfac16309d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea89b2d27cc04f3b8795bb7c46325d73'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:49.927321 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-bb595f37-f28f-416a-8437-b434c92bca67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '200347f85f274dd6bb321629a36f289b'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:49.941643 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-bae22248-5eaa-418a-8bc8-4dfac16309d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea89b2d27cc04f3b8795bb7c46325d73'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:49.949866 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-bb595f37-f28f-416a-8437-b434c92bca67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '200347f85f274dd6bb321629a36f289b'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:50.015173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-bae22248-5eaa-418a-8bc8-4dfac16309d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea89b2d27cc04f3b8795bb7c46325d73'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:50.015905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-bae22248-5eaa-418a-8bc8-4dfac16309d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea89b2d27cc04f3b8795bb7c46325d73'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:50.016512 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-bae22248-5eaa-418a-8bc8-4dfac16309d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea89b2d27cc04f3b8795bb7c46325d73'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:50.030137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 891/1798] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:50:49 2022] DELETE /identity/v3/users/ea89b2d27cc04f3b8795bb7c46325d73 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:50:50.034728 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b6bb5-eaf9-4544-9ea5-6c810cc83e7f None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:50.035061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b6bb5-eaf9-4544-9ea5-6c810cc83e7f None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:50.035318 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b6bb5-eaf9-4544-9ea5-6c810cc83e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:50.068050 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-bb7b6bb5-eaf9-4544-9ea5-6c810cc83e7f None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:50.069615 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-bb7b6bb5-eaf9-4544-9ea5-6c810cc83e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:50.102892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-bb595f37-f28f-416a-8437-b434c92bca67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '200347f85f274dd6bb321629a36f289b'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:50.103540 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-bb595f37-f28f-416a-8437-b434c92bca67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '200347f85f274dd6bb321629a36f289b'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:50.116067 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-bb595f37-f28f-416a-8437-b434c92bca67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '200347f85f274dd6bb321629a36f289b'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:50.137270 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 908/1799] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:50:49 2022] DELETE /identity/v3/users/200347f85f274dd6bb321629a36f289b => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:50.205031 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 892/1800] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:50:50.219116 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb595f37-f28f-416a-8437-b434c92bca67 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:50.220593 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-bb595f37-f28f-416a-8437-b434c92bca67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:50.303102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb595f37-f28f-416a-8437-b434c92bca67 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:50.308664 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af462a-10c0-407d-8a17-172298320b5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:50.309371 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af462a-10c0-407d-8a17-172298320b5f service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:50.309660 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af462a-10c0-407d-8a17-172298320b5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:50.309941 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af462a-10c0-407d-8a17-172298320b5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:50.443843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af462a-10c0-407d-8a17-172298320b5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:50.445898 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af462a-10c0-407d-8a17-172298320b5f service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:50.456781 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b6bb5-eaf9-4544-9ea5-6c810cc83e7f None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:50.458951 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-bb7b6bb5-eaf9-4544-9ea5-6c810cc83e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:50.465106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 909/1801] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:50.485154 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b6bb5-eaf9-4544-9ea5-6c810cc83e7f None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:50.490888 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e2854-fb6b-41b7-a985-3e3c72aafc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:50.492432 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e2854-fb6b-41b7-a985-3e3c72aafc14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:50.492946 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e2854-fb6b-41b7-a985-3e3c72aafc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:50.493386 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e2854-fb6b-41b7-a985-3e3c72aafc14 admin admin] PATH_INFO: `/v3/projects/07db11275ba743d4bdb6de057695dd22` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:50.494880 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e2854-fb6b-41b7-a985-3e3c72aafc14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07db11275ba743d4bdb6de057695dd22)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:50.497276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e2854-fb6b-41b7-a985-3e3c72aafc14 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:50.620566 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b18e2854-fb6b-41b7-a985-3e3c72aafc14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07db11275ba743d4bdb6de057695dd22', 'request_id': 'req-b18e2854-fb6b-41b7-a985-3e3c72aafc14'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:50.626905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b18e2854-fb6b-41b7-a985-3e3c72aafc14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07db11275ba743d4bdb6de057695dd22', 'request_id': 'req-b18e2854-fb6b-41b7-a985-3e3c72aafc14'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:50.736568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b18e2854-fb6b-41b7-a985-3e3c72aafc14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07db11275ba743d4bdb6de057695dd22', 'request_id': 'req-b18e2854-fb6b-41b7-a985-3e3c72aafc14'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:50.753455 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b18e2854-fb6b-41b7-a985-3e3c72aafc14 admin admin] The token cache is being invalidate because project 07db11275ba743d4bdb6de057695dd22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:50:50.754129 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b18e2854-fb6b-41b7-a985-3e3c72aafc14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:50.755040 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b18e2854-fb6b-41b7-a985-3e3c72aafc14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:50.756931 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 893/1802] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:50:50 2022] DELETE /identity/v3/projects/07db11275ba743d4bdb6de057695dd22 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:50.762811 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51788d4-796b-4d91-be27-ab3df9c40fbc None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:50.763109 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51788d4-796b-4d91-be27-ab3df9c40fbc None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:50.763514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51788d4-796b-4d91-be27-ab3df9c40fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:50.809452 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-c51788d4-796b-4d91-be27-ab3df9c40fbc None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:50.811953 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-c51788d4-796b-4d91-be27-ab3df9c40fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:50.928344 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 910/1803] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:50.931726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46809743-e0d1-4f1a-a9a3-b8b07e265889 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:50.932314 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46809743-e0d1-4f1a-a9a3-b8b07e265889 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:50.933470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46809743-e0d1-4f1a-a9a3-b8b07e265889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:50.993759 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-46809743-e0d1-4f1a-a9a3-b8b07e265889 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:50.995434 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-46809743-e0d1-4f1a-a9a3-b8b07e265889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:51.018851 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 894/1804] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:51.018966 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51788d4-796b-4d91-be27-ab3df9c40fbc None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:51.028893 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51788d4-796b-4d91-be27-ab3df9c40fbc None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:51.034699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549d491-742a-44b9-9647-7c66b52fd83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:51.035384 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549d491-742a-44b9-9647-7c66b52fd83c admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:51.035820 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549d491-742a-44b9-9647-7c66b52fd83c admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:51.036393 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549d491-742a-44b9-9647-7c66b52fd83c admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:51.037408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549d491-742a-44b9-9647-7c66b52fd83c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:50:51.037545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549d491-742a-44b9-9647-7c66b52fd83c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:51.039913 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549d491-742a-44b9-9647-7c66b52fd83c admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:51.051191 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:51.053281 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 911/1805] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:50:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:50:51.062448 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46809743-e0d1-4f1a-a9a3-b8b07e265889 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:51.078013 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46809743-e0d1-4f1a-a9a3-b8b07e265889 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:51.087264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf54b3-8765-4703-b01a-eb839590a96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:51.088347 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf54b3-8765-4703-b01a-eb839590a96c admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:51.088815 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf54b3-8765-4703-b01a-eb839590a96c admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:51.090066 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf54b3-8765-4703-b01a-eb839590a96c admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:51.091013 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf54b3-8765-4703-b01a-eb839590a96c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:51.094118 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cf54b3-8765-4703-b01a-eb839590a96c admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:51.135473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:51.195271 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 895/1806] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:51 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:51.200087 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549d491-742a-44b9-9647-7c66b52fd83c admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:51.210842 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549d491-742a-44b9-9647-7c66b52fd83c admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:51.273626 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738c8b0-fe4e-4149-bde7-e944ae0c6956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:51.274201 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738c8b0-fe4e-4149-bde7-e944ae0c6956 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:51.274637 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738c8b0-fe4e-4149-bde7-e944ae0c6956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:51.276983 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f738c8b0-fe4e-4149-bde7-e944ae0c6956 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:51.276983 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f738c8b0-fe4e-4149-bde7-e944ae0c6956 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:51.278748 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f738c8b0-fe4e-4149-bde7-e944ae0c6956 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:51.286118 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf54b3-8765-4703-b01a-eb839590a96c admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:51.288021 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-c4cf54b3-8765-4703-b01a-eb839590a96c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:51.309273 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf54b3-8765-4703-b01a-eb839590a96c admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:51.317106 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876927b-2ac8-430e-9257-5e7af150631f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:51.318477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876927b-2ac8-430e-9257-5e7af150631f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:51.318860 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876927b-2ac8-430e-9257-5e7af150631f admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:51.319217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7876927b-2ac8-430e-9257-5e7af150631f admin admin] PATH_INFO: `/v3/projects/bce1d5722fee4bf892aa29ad12a0e4f7` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:51.321469 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876927b-2ac8-430e-9257-5e7af150631f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bce1d5722fee4bf892aa29ad12a0e4f7)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:51.323829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7876927b-2ac8-430e-9257-5e7af150631f admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:51.362791 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 912/1807] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:51 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:51.364175 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738c8b0-fe4e-4149-bde7-e944ae0c6956 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:51.374318 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f738c8b0-fe4e-4149-bde7-e944ae0c6956 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:51.379279 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a14112-0da1-4eca-a428-b4a0cb0a47e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:51.380017 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a14112-0da1-4eca-a428-b4a0cb0a47e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:51.380374 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a14112-0da1-4eca-a428-b4a0cb0a47e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:51.380859 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a14112-0da1-4eca-a428-b4a0cb0a47e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:51.381623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a14112-0da1-4eca-a428-b4a0cb0a47e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:51.383913 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a14112-0da1-4eca-a428-b4a0cb0a47e1 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:51.399278 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 913/1808] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:50:51.400178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a14112-0da1-4eca-a428-b4a0cb0a47e1 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:51.407106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-79a14112-0da1-4eca-a428-b4a0cb0a47e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:51.430108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a14112-0da1-4eca-a428-b4a0cb0a47e1 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:51.436712 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d89e3e-ff1b-4a12-8a62-422da3a7680d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:51.437717 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d89e3e-ff1b-4a12-8a62-422da3a7680d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:51.438319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d89e3e-ff1b-4a12-8a62-422da3a7680d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:51.438603 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d89e3e-ff1b-4a12-8a62-422da3a7680d admin admin] PATH_INFO: `/v3/projects/4520fe04426a43a3b9fb356247e4fae9` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:51.440079 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d89e3e-ff1b-4a12-8a62-422da3a7680d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4520fe04426a43a3b9fb356247e4fae9)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:51.442077 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d89e3e-ff1b-4a12-8a62-422da3a7680d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:51.521929 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-7876927b-2ac8-430e-9257-5e7af150631f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bce1d5722fee4bf892aa29ad12a0e4f7', 'request_id': 'req-7876927b-2ac8-430e-9257-5e7af150631f'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:51.530995 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-7876927b-2ac8-430e-9257-5e7af150631f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bce1d5722fee4bf892aa29ad12a0e4f7', 'request_id': 'req-7876927b-2ac8-430e-9257-5e7af150631f'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:51.629334 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-18d89e3e-ff1b-4a12-8a62-422da3a7680d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4520fe04426a43a3b9fb356247e4fae9', 'request_id': 'req-18d89e3e-ff1b-4a12-8a62-422da3a7680d'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:51.635490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-18d89e3e-ff1b-4a12-8a62-422da3a7680d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4520fe04426a43a3b9fb356247e4fae9', 'request_id': 'req-18d89e3e-ff1b-4a12-8a62-422da3a7680d'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:51.642522 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-18d89e3e-ff1b-4a12-8a62-422da3a7680d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4520fe04426a43a3b9fb356247e4fae9', 'request_id': 'req-18d89e3e-ff1b-4a12-8a62-422da3a7680d'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:51.644992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-7876927b-2ac8-430e-9257-5e7af150631f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bce1d5722fee4bf892aa29ad12a0e4f7', 'request_id': 'req-7876927b-2ac8-430e-9257-5e7af150631f'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:51.660874 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-7876927b-2ac8-430e-9257-5e7af150631f admin admin] The token cache is being invalidate because project bce1d5722fee4bf892aa29ad12a0e4f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:50:51.661201 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-7876927b-2ac8-430e-9257-5e7af150631f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:51.661895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-7876927b-2ac8-430e-9257-5e7af150631f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:51.664593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 896/1809] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:50:51 2022] DELETE /identity/v3/projects/bce1d5722fee4bf892aa29ad12a0e4f7 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:50:51.665924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876927b-2ac8-430e-9257-5e7af150631f admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:51.667442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-7876927b-2ac8-430e-9257-5e7af150631f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:51.733175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7876927b-2ac8-430e-9257-5e7af150631f admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:51.787470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-18d89e3e-ff1b-4a12-8a62-422da3a7680d admin admin] The token cache is being invalidate because project 4520fe04426a43a3b9fb356247e4fae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:50:51.787698 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-18d89e3e-ff1b-4a12-8a62-422da3a7680d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:51.788340 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-18d89e3e-ff1b-4a12-8a62-422da3a7680d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:51.789097 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e059a7-e069-456f-b01c-035590157e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:51.789883 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e059a7-e069-456f-b01c-035590157e58 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:51.790171 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e059a7-e069-456f-b01c-035590157e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:51.790448 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e059a7-e069-456f-b01c-035590157e58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:51.790723 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 914/1810] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:50:51 2022] DELETE /identity/v3/projects/4520fe04426a43a3b9fb356247e4fae9 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:51.791251 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e059a7-e069-456f-b01c-035590157e58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:51.792090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f1a4b-2d43-442c-a09f-d610b0d03179 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:51.792382 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f1a4b-2d43-442c-a09f-d610b0d03179 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:51.792663 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858f1a4b-2d43-442c-a09f-d610b0d03179 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:51.792852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e059a7-e069-456f-b01c-035590157e58 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:51.794415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 915/1811] 10.222.0.13 () {54 vars in 921 bytes} [Fri May 27 19:50:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:50:51.795441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e9e9b-8567-4571-92c4-ad4c5c20b909 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:51.795792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e9e9b-8567-4571-92c4-ad4c5c20b909 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:51.796090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e9e9b-8567-4571-92c4-ad4c5c20b909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:51.803956 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 897/1812] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:51.805140 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129062eb-b958-4013-81f0-46b1c3503792 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:51.805435 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129062eb-b958-4013-81f0-46b1c3503792 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:51.805720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129062eb-b958-4013-81f0-46b1c3503792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:51.817584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-902e9e9b-8567-4571-92c4-ad4c5c20b909 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:51.818798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-902e9e9b-8567-4571-92c4-ad4c5c20b909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:51.823836 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-129062eb-b958-4013-81f0-46b1c3503792 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:51.825153 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-129062eb-b958-4013-81f0-46b1c3503792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:51.826863 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 916/1813] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:51.828182 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a129a4-de78-4b41-92f7-709ee28c4ee6 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:51.828478 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a129a4-de78-4b41-92f7-709ee28c4ee6 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:51.828758 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a129a4-de78-4b41-92f7-709ee28c4ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:51.833877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 898/1814] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:51.841501 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129062eb-b958-4013-81f0-46b1c3503792 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:51.841501 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-129062eb-b958-4013-81f0-46b1c3503792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:51.848109 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-95a129a4-de78-4b41-92f7-709ee28c4ee6 None None] MFA Rules not processed for user `7542e1fd160e4b2c9d25accf19aad19f`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:51.849298 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129062eb-b958-4013-81f0-46b1c3503792 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:51.849406 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-95a129a4-de78-4b41-92f7-709ee28c4ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:51.855100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a9a08-1715-4a77-8706-7ddee8b3fdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:51.856008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a9a08-1715-4a77-8706-7ddee8b3fdf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:51.856288 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a9a08-1715-4a77-8706-7ddee8b3fdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:51.856563 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a9a08-1715-4a77-8706-7ddee8b3fdf6 admin admin] PATH_INFO: `/v3/projects/f22a0940d9a24f4e940313e1f5e55e8d/users/a819d84211dc4cc7a0d21664378e7b70/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:51.871938 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a9a08-1715-4a77-8706-7ddee8b3fdf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f22a0940d9a24f4e940313e1f5e55e8d, user_id=a819d84211dc4cc7a0d21664378e7b70, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:51.876787 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a9a08-1715-4a77-8706-7ddee8b3fdf6 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:51.917175 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 917/1815] 10.222.0.13 () {58 vars in 1018 bytes} [Fri May 27 19:50:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:51.918571 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da6d13-082e-4999-b9e4-082624f1e80e None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:51.918869 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da6d13-082e-4999-b9e4-082624f1e80e None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:51.919153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da6d13-082e-4999-b9e4-082624f1e80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:51.937001 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-b8da6d13-082e-4999-b9e4-082624f1e80e None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:51.938221 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-b8da6d13-082e-4999-b9e4-082624f1e80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:51.946720 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 918/1816] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:51.948153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62971706-c274-4f48-9f92-1afd8f7df461 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:51.948439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62971706-c274-4f48-9f92-1afd8f7df461 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:51.948720 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62971706-c274-4f48-9f92-1afd8f7df461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:51.959607 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 899/1817] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:51 2022] PUT /identity/v3/projects/f22a0940d9a24f4e940313e1f5e55e8d/users/a819d84211dc4cc7a0d21664378e7b70/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:51.960568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a9a08-1715-4a77-8706-7ddee8b3fdf6 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:51.968845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-230a9a08-1715-4a77-8706-7ddee8b3fdf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:51.969120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-62971706-c274-4f48-9f92-1afd8f7df461 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:51.969977 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-62971706-c274-4f48-9f92-1afd8f7df461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:52.029149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 919/1818] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:52.030007 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62971706-c274-4f48-9f92-1afd8f7df461 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:52.036439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a9a08-1715-4a77-8706-7ddee8b3fdf6 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:52.039424 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62971706-c274-4f48-9f92-1afd8f7df461 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:52.042125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e4a77-01f9-4726-be7b-b06498009d76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:52.042783 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e4a77-01f9-4726-be7b-b06498009d76 service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:52.043072 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e4a77-01f9-4726-be7b-b06498009d76 service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:52.043351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e4a77-01f9-4726-be7b-b06498009d76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:52.044498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09dc30e-8612-4d30-b8d6-1e524f1897e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:52.045177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09dc30e-8612-4d30-b8d6-1e524f1897e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:52.045476 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09dc30e-8612-4d30-b8d6-1e524f1897e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:52.045774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09dc30e-8612-4d30-b8d6-1e524f1897e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:52.046494 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09dc30e-8612-4d30-b8d6-1e524f1897e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:50:52.046831 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09dc30e-8612-4d30-b8d6-1e524f1897e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:52.048446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09dc30e-8612-4d30-b8d6-1e524f1897e1 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:52.052212 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e4a77-01f9-4726-be7b-b06498009d76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:52.055448 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e4a77-01f9-4726-be7b-b06498009d76 service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:52.060453 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 920/1819] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:50:52.061908 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c872f0-d012-47da-8e5f-91695b32c14a None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:52.062200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c872f0-d012-47da-8e5f-91695b32c14a None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:52.062479 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c872f0-d012-47da-8e5f-91695b32c14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:52.064699 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 900/1820] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:52.065565 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e4a77-01f9-4726-be7b-b06498009d76 service nova] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:52.076398 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e4a77-01f9-4726-be7b-b06498009d76 service nova] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:52.081210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c39b1-238c-4c31-af84-7b6deaa3b3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:52.081878 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c39b1-238c-4c31-af84-7b6deaa3b3d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:52.082166 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c39b1-238c-4c31-af84-7b6deaa3b3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:52.082451 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c39b1-238c-4c31-af84-7b6deaa3b3d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:52.083191 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c39b1-238c-4c31-af84-7b6deaa3b3d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:50:52.083495 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c39b1-238c-4c31-af84-7b6deaa3b3d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:52.085138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690c39b1-238c-4c31-af84-7b6deaa3b3d8 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:52.090839 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-10c872f0-d012-47da-8e5f-91695b32c14a None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:52.095799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-10c872f0-d012-47da-8e5f-91695b32c14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:52.098446 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 901/1821] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:50:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:50:52.102888 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c39b1-238c-4c31-af84-7b6deaa3b3d8 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:52.105090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 921/1822] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:52.105908 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c872f0-d012-47da-8e5f-91695b32c14a None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:52.118261 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690c39b1-238c-4c31-af84-7b6deaa3b3d8 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:52.118378 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c872f0-d012-47da-8e5f-91695b32c14a None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:52.120992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc4e27-4cfe-4bf8-b337-e580102708f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:52.121777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc4e27-4cfe-4bf8-b337-e580102708f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:52.122068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc4e27-4cfe-4bf8-b337-e580102708f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:52.122470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc4e27-4cfe-4bf8-b337-e580102708f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:52.122644 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e4c78-7fc6-4f36-9c27-9e7acc919378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:52.123101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc4e27-4cfe-4bf8-b337-e580102708f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:52.123252 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e4c78-7fc6-4f36-9c27-9e7acc919378 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:52.123542 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e4c78-7fc6-4f36-9c27-9e7acc919378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:52.123819 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e4c78-7fc6-4f36-9c27-9e7acc919378 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:52.124511 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e4c78-7fc6-4f36-9c27-9e7acc919378 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:52.125030 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fc4e27-4cfe-4bf8-b337-e580102708f1 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:52.126597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e4c78-7fc6-4f36-9c27-9e7acc919378 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:52.207021 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 922/1823] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:52 2022] POST /identity/v3/projects => generated 405 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:52.207883 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e4c78-7fc6-4f36-9c27-9e7acc919378 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:52.209851 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ac2e4c78-7fc6-4f36-9c27-9e7acc919378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:52.220103 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 902/1824] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:52 2022] POST /identity/v3/projects => generated 387 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:52.220930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc4e27-4cfe-4bf8-b337-e580102708f1 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:52.268298 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:52.278432 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e4c78-7fc6-4f36-9c27-9e7acc919378 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:52.303941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fc4e27-4cfe-4bf8-b337-e580102708f1 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:52.310103 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6166e-db1a-4ecc-a7f7-3cac7348b544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:52.310808 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6166e-db1a-4ecc-a7f7-3cac7348b544 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:52.311094 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6166e-db1a-4ecc-a7f7-3cac7348b544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:52.311469 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6166e-db1a-4ecc-a7f7-3cac7348b544 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:52.312127 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6166e-db1a-4ecc-a7f7-3cac7348b544 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:52.314224 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6166e-db1a-4ecc-a7f7-3cac7348b544 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:52.315195 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:52.335248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448d3bc-ca1e-4181-b50c-3d2cfa88382c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:52.335910 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448d3bc-ca1e-4181-b50c-3d2cfa88382c service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:52.336191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448d3bc-ca1e-4181-b50c-3d2cfa88382c service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:52.337056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448d3bc-ca1e-4181-b50c-3d2cfa88382c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:52.346388 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d448d3bc-ca1e-4181-b50c-3d2cfa88382c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:52.347829 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d448d3bc-ca1e-4181-b50c-3d2cfa88382c service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:52.357551 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 923/1825] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:52.358188 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448d3bc-ca1e-4181-b50c-3d2cfa88382c service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:52.367259 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448d3bc-ca1e-4181-b50c-3d2cfa88382c service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:52.373620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af2682-962a-4737-8d16-06a187758b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:52.374346 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af2682-962a-4737-8d16-06a187758b8d admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:52.374640 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af2682-962a-4737-8d16-06a187758b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:52.374937 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af2682-962a-4737-8d16-06a187758b8d admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:52.375627 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af2682-962a-4737-8d16-06a187758b8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:52.377618 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af2682-962a-4737-8d16-06a187758b8d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:52.409148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 903/1826] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:52 2022] POST /identity/v3/users => generated 303 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:52.413584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6166e-db1a-4ecc-a7f7-3cac7348b544 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:52.424509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6166e-db1a-4ecc-a7f7-3cac7348b544 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:52.429876 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57840f02-2f1a-469e-a980-9b9778cd3245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:52.430705 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57840f02-2f1a-469e-a980-9b9778cd3245 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:52.430999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57840f02-2f1a-469e-a980-9b9778cd3245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:52.431353 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57840f02-2f1a-469e-a980-9b9778cd3245 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:52.432091 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57840f02-2f1a-469e-a980-9b9778cd3245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:52.433830 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57840f02-2f1a-469e-a980-9b9778cd3245 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:52.444037 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 904/1827] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:50:52.448386 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57840f02-2f1a-469e-a980-9b9778cd3245 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:52.457913 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57840f02-2f1a-469e-a980-9b9778cd3245 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:52.462950 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43220194-156f-46ed-a52f-2d30b657ebd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:52.463616 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43220194-156f-46ed-a52f-2d30b657ebd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:52.463905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43220194-156f-46ed-a52f-2d30b657ebd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:52.464176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43220194-156f-46ed-a52f-2d30b657ebd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:52.465006 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43220194-156f-46ed-a52f-2d30b657ebd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:52.466499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43220194-156f-46ed-a52f-2d30b657ebd7 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:52.482055 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 905/1828] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:50:52.486144 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43220194-156f-46ed-a52f-2d30b657ebd7 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:52.496467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43220194-156f-46ed-a52f-2d30b657ebd7 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:52.501255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50ddfe-2702-4de5-90be-48b07640193a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:52.502158 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50ddfe-2702-4de5-90be-48b07640193a admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:52.502451 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50ddfe-2702-4de5-90be-48b07640193a admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:52.502731 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50ddfe-2702-4de5-90be-48b07640193a admin admin] PATH_INFO: `/v3/projects/5bbf9e5e42a949fb892948517a8f7f81/users/70d94c4f050a4dadb0a7c147358fcfa9/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:52.520195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50ddfe-2702-4de5-90be-48b07640193a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bbf9e5e42a949fb892948517a8f7f81, user_id=70d94c4f050a4dadb0a7c147358fcfa9, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:52.523065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50ddfe-2702-4de5-90be-48b07640193a admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:52.528815 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 924/1829] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:52 2022] POST /identity/v3/users => generated 294 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:52.530231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af2682-962a-4737-8d16-06a187758b8d admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:52.543830 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af2682-962a-4737-8d16-06a187758b8d admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:52.569491 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1089e860-7094-41ff-ad4f-3676627fcbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:52.570328 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089e860-7094-41ff-ad4f-3676627fcbeb admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:52.570732 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089e860-7094-41ff-ad4f-3676627fcbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:52.571085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089e860-7094-41ff-ad4f-3676627fcbeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:52.571821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1089e860-7094-41ff-ad4f-3676627fcbeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:52.573486 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1089e860-7094-41ff-ad4f-3676627fcbeb admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:52.601386 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 925/1830] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:50:52.603577 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1089e860-7094-41ff-ad4f-3676627fcbeb admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:52.615081 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1089e860-7094-41ff-ad4f-3676627fcbeb admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:52.625219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe6f7a-be25-4019-8479-369fa8b7ea09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:52.625219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe6f7a-be25-4019-8479-369fa8b7ea09 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:52.625219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe6f7a-be25-4019-8479-369fa8b7ea09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:52.625219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe6f7a-be25-4019-8479-369fa8b7ea09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:52.625583 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe6f7a-be25-4019-8479-369fa8b7ea09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:52.625583 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe6f7a-be25-4019-8479-369fa8b7ea09 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:52.640343 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 926/1831] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:50:52.641613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe6f7a-be25-4019-8479-369fa8b7ea09 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:52.653414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe6f7a-be25-4019-8479-369fa8b7ea09 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:52.662217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 906/1832] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:52 2022] PUT /identity/v3/projects/5bbf9e5e42a949fb892948517a8f7f81/users/70d94c4f050a4dadb0a7c147358fcfa9/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:50:52.674018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b561a69-a40b-439d-aceb-1684cf946e4b None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:52.674647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b561a69-a40b-439d-aceb-1684cf946e4b None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:52.675191 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b561a69-a40b-439d-aceb-1684cf946e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:52.713941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-5b561a69-a40b-439d-aceb-1684cf946e4b None None] MFA Rules not processed for user `70d94c4f050a4dadb0a7c147358fcfa9`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:52.717541 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-5b561a69-a40b-439d-aceb-1684cf946e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:52.759690 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564299f1-9f69-4d7f-90cb-81c373212e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:52.759690 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564299f1-9f69-4d7f-90cb-81c373212e21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:52.759690 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564299f1-9f69-4d7f-90cb-81c373212e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:52.760161 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564299f1-9f69-4d7f-90cb-81c373212e21 admin admin] PATH_INFO: `/v3/projects/7abae1a562d94cc49e3f9845aa890646/users/26c84d3f1b8e46bebf02c29a8591adb1/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:52.777778 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564299f1-9f69-4d7f-90cb-81c373212e21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7abae1a562d94cc49e3f9845aa890646, user_id=26c84d3f1b8e46bebf02c29a8591adb1, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:52.781199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564299f1-9f69-4d7f-90cb-81c373212e21 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:52.846516 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 907/1833] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:52 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:50:52.850542 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b561a69-a40b-439d-aceb-1684cf946e4b None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:52.862125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b561a69-a40b-439d-aceb-1684cf946e4b None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:52.867002 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f48c7-1605-4511-abdd-914c93315a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:52.867852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f48c7-1605-4511-abdd-914c93315a12 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:52.868230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f48c7-1605-4511-abdd-914c93315a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:52.868537 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f48c7-1605-4511-abdd-914c93315a12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:52.869421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f48c7-1605-4511-abdd-914c93315a12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:52.871494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f48c7-1605-4511-abdd-914c93315a12 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:52.895848 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 927/1834] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:52 2022] PUT /identity/v3/projects/7abae1a562d94cc49e3f9845aa890646/users/26c84d3f1b8e46bebf02c29a8591adb1/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:52.906832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82e20b-c3cd-4763-acc7-bf6dbb229afb None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:52.907903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82e20b-c3cd-4763-acc7-bf6dbb229afb None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:52.908552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82e20b-c3cd-4763-acc7-bf6dbb229afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:52.957686 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-4c82e20b-c3cd-4763-acc7-bf6dbb229afb None None] MFA Rules not processed for user `26c84d3f1b8e46bebf02c29a8591adb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:52.962112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-4c82e20b-c3cd-4763-acc7-bf6dbb229afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:52.988925 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 908/1835] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:52 2022] POST /identity/v3/projects => generated 401 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:50:52.993690 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f48c7-1605-4511-abdd-914c93315a12 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:53.064368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 928/1836] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:50:53.065818 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82e20b-c3cd-4763-acc7-bf6dbb229afb None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:53.081348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f48c7-1605-4511-abdd-914c93315a12 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:53.087214 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843106ef-d9a8-4ca7-9f73-682a950504e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:53.088161 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843106ef-d9a8-4ca7-9f73-682a950504e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:53.088451 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843106ef-d9a8-4ca7-9f73-682a950504e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:53.088727 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843106ef-d9a8-4ca7-9f73-682a950504e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:53.089543 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843106ef-d9a8-4ca7-9f73-682a950504e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:53.091740 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843106ef-d9a8-4ca7-9f73-682a950504e2 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:53.141201 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82e20b-c3cd-4763-acc7-bf6dbb229afb None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:53.147238 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37258a9e-e31c-43b2-9bf7-2a495eef1f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:53.148001 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37258a9e-e31c-43b2-9bf7-2a495eef1f0f admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:53.148384 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37258a9e-e31c-43b2-9bf7-2a495eef1f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:53.149344 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37258a9e-e31c-43b2-9bf7-2a495eef1f0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:53.150243 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37258a9e-e31c-43b2-9bf7-2a495eef1f0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:53.152717 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37258a9e-e31c-43b2-9bf7-2a495eef1f0f admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:53.219041 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 909/1837] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:52 2022] POST /identity/v3/users => generated 307 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:53.222967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843106ef-d9a8-4ca7-9f73-682a950504e2 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:53.240296 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843106ef-d9a8-4ca7-9f73-682a950504e2 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:53.243036 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 929/1838] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:53 2022] POST /identity/v3/projects => generated 387 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:53.250690 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37258a9e-e31c-43b2-9bf7-2a495eef1f0f admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:53.251778 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdda2f4-1c70-4f37-81ec-abd0f8e98dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:53.252822 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdda2f4-1c70-4f37-81ec-abd0f8e98dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:53.252822 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdda2f4-1c70-4f37-81ec-abd0f8e98dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:53.253043 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdda2f4-1c70-4f37-81ec-abd0f8e98dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:53.253772 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdda2f4-1c70-4f37-81ec-abd0f8e98dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:53.256913 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdda2f4-1c70-4f37-81ec-abd0f8e98dd4 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:53.274480 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 910/1839] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:50:53.275410 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdda2f4-1c70-4f37-81ec-abd0f8e98dd4 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:53.292398 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdda2f4-1c70-4f37-81ec-abd0f8e98dd4 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:53.356639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37258a9e-e31c-43b2-9bf7-2a495eef1f0f admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:53.362950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede6e105-a1cd-4a71-a654-38c6d2b6a88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:53.363849 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6e105-a1cd-4a71-a654-38c6d2b6a88b admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:53.364393 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6e105-a1cd-4a71-a654-38c6d2b6a88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:53.364820 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6e105-a1cd-4a71-a654-38c6d2b6a88b admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:53.365693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede6e105-a1cd-4a71-a654-38c6d2b6a88b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:53.368001 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede6e105-a1cd-4a71-a654-38c6d2b6a88b admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:53.390974 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2940ea0-4822-4b5e-946f-3ce14b098fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:53.392206 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2940ea0-4822-4b5e-946f-3ce14b098fa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:53.392607 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2940ea0-4822-4b5e-946f-3ce14b098fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:53.393012 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2940ea0-4822-4b5e-946f-3ce14b098fa0 admin admin] PATH_INFO: `/v3/projects/b78da24e53e24dcf8218e07400955778/users/53d3aba4f9c24c80be4277049e342220/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:53.413569 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2940ea0-4822-4b5e-946f-3ce14b098fa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b78da24e53e24dcf8218e07400955778, user_id=53d3aba4f9c24c80be4277049e342220, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:53.418079 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2940ea0-4822-4b5e-946f-3ce14b098fa0 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:53.474999 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 930/1840] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:53 2022] POST /identity/v3/users => generated 300 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:53.479321 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede6e105-a1cd-4a71-a654-38c6d2b6a88b admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:53.493619 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:53.494136 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede6e105-a1cd-4a71-a654-38c6d2b6a88b admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:53.513033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 911/1841] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:53 2022] PUT /identity/v3/projects/b78da24e53e24dcf8218e07400955778/users/53d3aba4f9c24c80be4277049e342220/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:53.516789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2940ea0-4822-4b5e-946f-3ce14b098fa0 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:53.635519 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d8a2b-b537-4926-b8b7-81fc09ddb1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:53.637003 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d8a2b-b537-4926-b8b7-81fc09ddb1fe admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:53.637534 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d8a2b-b537-4926-b8b7-81fc09ddb1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:53.638025 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d8a2b-b537-4926-b8b7-81fc09ddb1fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:53.639375 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d8a2b-b537-4926-b8b7-81fc09ddb1fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:53.642314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d8a2b-b537-4926-b8b7-81fc09ddb1fe admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:53.643312 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2940ea0-4822-4b5e-946f-3ce14b098fa0 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:53.648430 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb5e13-fb8c-4d00-a45e-f376f7f8bb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:53.650261 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb5e13-fb8c-4d00-a45e-f376f7f8bb62 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:53.650624 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb5e13-fb8c-4d00-a45e-f376f7f8bb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:53.650914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb5e13-fb8c-4d00-a45e-f376f7f8bb62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:53.652071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb5e13-fb8c-4d00-a45e-f376f7f8bb62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:53.654163 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb5e13-fb8c-4d00-a45e-f376f7f8bb62 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:53.666286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 931/1842] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:50:53.667483 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 912/1843] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:53.670421 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d8a2b-b537-4926-b8b7-81fc09ddb1fe admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:53.674304 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb5e13-fb8c-4d00-a45e-f376f7f8bb62 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:53.683059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d8a2b-b537-4926-b8b7-81fc09ddb1fe admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:53.687315 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb5e13-fb8c-4d00-a45e-f376f7f8bb62 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:53.688898 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d49e0b-ffe1-4039-ba78-6a111c47fb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:53.689863 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d49e0b-ffe1-4039-ba78-6a111c47fb03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:53.690146 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d49e0b-ffe1-4039-ba78-6a111c47fb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:53.690427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d49e0b-ffe1-4039-ba78-6a111c47fb03 admin admin] PATH_INFO: `/v3/projects/c5bc5d4177f74942ae15db1629dbb7d5/users/70fb17b1774f4b738b45411c27dfd0e2/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:53.694434 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f922ad-7dfd-40e0-9465-70c5335f93c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:53.696353 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f922ad-7dfd-40e0-9465-70c5335f93c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:53.696905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f922ad-7dfd-40e0-9465-70c5335f93c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:53.697530 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:53.697660 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f922ad-7dfd-40e0-9465-70c5335f93c0 admin admin] PATH_INFO: `/v3/projects/b78da24e53e24dcf8218e07400955778/users/53d3aba4f9c24c80be4277049e342220/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:53.700250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f922ad-7dfd-40e0-9465-70c5335f93c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b78da24e53e24dcf8218e07400955778, user_id=53d3aba4f9c24c80be4277049e342220, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:53.705400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f922ad-7dfd-40e0-9465-70c5335f93c0 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:53.710314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d49e0b-ffe1-4039-ba78-6a111c47fb03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5bc5d4177f74942ae15db1629dbb7d5, user_id=70fb17b1774f4b738b45411c27dfd0e2, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:53.719637 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d49e0b-ffe1-4039-ba78-6a111c47fb03 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:53.732627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 913/1844] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:53 2022] PUT /identity/v3/projects/b78da24e53e24dcf8218e07400955778/users/53d3aba4f9c24c80be4277049e342220/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:53.745410 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d0771-2380-4fde-9b3f-cfdfca7d500a None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:53.745673 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d0771-2380-4fde-9b3f-cfdfca7d500a None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:53.745959 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d0771-2380-4fde-9b3f-cfdfca7d500a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:53.776222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-e87d0771-2380-4fde-9b3f-cfdfca7d500a None None] MFA Rules not processed for user `53d3aba4f9c24c80be4277049e342220`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:53.777574 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-e87d0771-2380-4fde-9b3f-cfdfca7d500a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:53.806038 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 932/1845] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:53 2022] PUT /identity/v3/projects/c5bc5d4177f74942ae15db1629dbb7d5/users/70fb17b1774f4b738b45411c27dfd0e2/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:53.807113 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d49e0b-ffe1-4039-ba78-6a111c47fb03 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:53.885177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d49e0b-ffe1-4039-ba78-6a111c47fb03 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:53.893008 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ceb00-bd70-417c-ae08-eac20e108af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:53.893008 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ceb00-bd70-417c-ae08-eac20e108af3 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:53.893008 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ceb00-bd70-417c-ae08-eac20e108af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:53.893008 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ceb00-bd70-417c-ae08-eac20e108af3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:53.893454 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ceb00-bd70-417c-ae08-eac20e108af3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:53.895162 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ceb00-bd70-417c-ae08-eac20e108af3 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:53.907152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 933/1846] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:53 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:53.911369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ceb00-bd70-417c-ae08-eac20e108af3 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:53.921224 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 914/1847] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:53 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:53.927783 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ceb00-bd70-417c-ae08-eac20e108af3 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:53.934096 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d0771-2380-4fde-9b3f-cfdfca7d500a None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:53.935700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-e87d0771-2380-4fde-9b3f-cfdfca7d500a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:53.939997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf0948-83cf-45c3-b7c9-d07880fe2294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:53.941456 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf0948-83cf-45c3-b7c9-d07880fe2294 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:53.941965 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf0948-83cf-45c3-b7c9-d07880fe2294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:53.943317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf0948-83cf-45c3-b7c9-d07880fe2294 admin admin] PATH_INFO: `/v3/projects/c5bc5d4177f74942ae15db1629dbb7d5/users/70fb17b1774f4b738b45411c27dfd0e2/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:53.945462 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf0948-83cf-45c3-b7c9-d07880fe2294 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5bc5d4177f74942ae15db1629dbb7d5, user_id=70fb17b1774f4b738b45411c27dfd0e2, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:53.949484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf0948-83cf-45c3-b7c9-d07880fe2294 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:53.977509 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 934/1848] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:53 2022] PUT /identity/v3/projects/c5bc5d4177f74942ae15db1629dbb7d5/users/70fb17b1774f4b738b45411c27dfd0e2/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:53.989755 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00602c-b6a0-4daa-8526-80950c6beac1 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:53.990050 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00602c-b6a0-4daa-8526-80950c6beac1 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:53.990331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00602c-b6a0-4daa-8526-80950c6beac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:54.024793 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d0771-2380-4fde-9b3f-cfdfca7d500a None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:54.052343 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-cb00602c-b6a0-4daa-8526-80950c6beac1 None None] MFA Rules not processed for user `70fb17b1774f4b738b45411c27dfd0e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:54.054386 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-cb00602c-b6a0-4daa-8526-80950c6beac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:54.123119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da4bb2-4d23-48b0-8ef3-3b3241eeb7d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:54.124015 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da4bb2-4d23-48b0-8ef3-3b3241eeb7d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:54.124455 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da4bb2-4d23-48b0-8ef3-3b3241eeb7d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:54.124730 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da4bb2-4d23-48b0-8ef3-3b3241eeb7d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:54.197149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 935/1849] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:50:54.231754 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00602c-b6a0-4daa-8526-80950c6beac1 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:54.232825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-cb00602c-b6a0-4daa-8526-80950c6beac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:54.233345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da4bb2-4d23-48b0-8ef3-3b3241eeb7d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:54.234827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da4bb2-4d23-48b0-8ef3-3b3241eeb7d8 service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:54.246932 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00602c-b6a0-4daa-8526-80950c6beac1 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:54.248530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 915/1850] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:53 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:50:54.254811 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3266c419-7fac-4fc5-89d7-835b3260f6fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:54.254811 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266c419-7fac-4fc5-89d7-835b3260f6fe service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:54.254811 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266c419-7fac-4fc5-89d7-835b3260f6fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:54.254811 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3266c419-7fac-4fc5-89d7-835b3260f6fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:54.267385 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266c419-7fac-4fc5-89d7-835b3260f6fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:54.269404 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3266c419-7fac-4fc5-89d7-835b3260f6fe service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:54.281540 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 936/1851] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:50:54.415314 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da4bb2-4d23-48b0-8ef3-3b3241eeb7d8 service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:54.481097 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da4bb2-4d23-48b0-8ef3-3b3241eeb7d8 service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:54.486085 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd62f3-3db5-4263-abda-2d811c227c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:54.487016 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd62f3-3db5-4263-abda-2d811c227c14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:54.487306 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd62f3-3db5-4263-abda-2d811c227c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:54.487581 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd62f3-3db5-4263-abda-2d811c227c14 admin admin] PATH_INFO: `/v3/users/70d94c4f050a4dadb0a7c147358fcfa9` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:54.488766 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdd62f3-3db5-4263-abda-2d811c227c14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70d94c4f050a4dadb0a7c147358fcfa9)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:54.490730 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdd62f3-3db5-4263-abda-2d811c227c14 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:54.709316 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a66559-0f96-46b0-a402-f4c6f15e4758 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:54.709759 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a66559-0f96-46b0-a402-f4c6f15e4758 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:54.709929 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a66559-0f96-46b0-a402-f4c6f15e4758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:54.737138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-afdd62f3-3db5-4263-abda-2d811c227c14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70d94c4f050a4dadb0a7c147358fcfa9'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:54.750168 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-a9a66559-0f96-46b0-a402-f4c6f15e4758 None None] MFA Rules not processed for user `a819d84211dc4cc7a0d21664378e7b70`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:54.751395 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-a9a66559-0f96-46b0-a402-f4c6f15e4758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:54.757651 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-afdd62f3-3db5-4263-abda-2d811c227c14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70d94c4f050a4dadb0a7c147358fcfa9'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:54.850887 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-afdd62f3-3db5-4263-abda-2d811c227c14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70d94c4f050a4dadb0a7c147358fcfa9'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:54.851552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-afdd62f3-3db5-4263-abda-2d811c227c14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70d94c4f050a4dadb0a7c147358fcfa9'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:54.852321 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-afdd62f3-3db5-4263-abda-2d811c227c14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70d94c4f050a4dadb0a7c147358fcfa9'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:54.854781 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 937/1852] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:50:54.855883 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a66559-0f96-46b0-a402-f4c6f15e4758 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:54.870794 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 916/1853] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:50:54 2022] DELETE /identity/v3/users/70d94c4f050a4dadb0a7c147358fcfa9 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:54.872721 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280e7a3-6fc0-46a1-b618-d281090140cf None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:54.872837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a66559-0f96-46b0-a402-f4c6f15e4758 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:54.876513 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280e7a3-6fc0-46a1-b618-d281090140cf None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:54.876513 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280e7a3-6fc0-46a1-b618-d281090140cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:54.892062 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-e280e7a3-6fc0-46a1-b618-d281090140cf None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:54.893593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-e280e7a3-6fc0-46a1-b618-d281090140cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:54.980259 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e0e1a-63e8-41ee-80ff-c77f11e109cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:54.980978 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e0e1a-63e8-41ee-80ff-c77f11e109cc admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:54.981320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e0e1a-63e8-41ee-80ff-c77f11e109cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:54.981594 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e0e1a-63e8-41ee-80ff-c77f11e109cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:54.982308 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4e0e1a-63e8-41ee-80ff-c77f11e109cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:54.989225 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4e0e1a-63e8-41ee-80ff-c77f11e109cc admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:54.999656 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 917/1854] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:55.015681 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280e7a3-6fc0-46a1-b618-d281090140cf None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:55.018428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-e280e7a3-6fc0-46a1-b618-d281090140cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:55.101575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280e7a3-6fc0-46a1-b618-d281090140cf None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:55.101575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f6d28-23c9-4874-aebd-9252ca325483 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:55.101575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f6d28-23c9-4874-aebd-9252ca325483 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:55.101575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f6d28-23c9-4874-aebd-9252ca325483 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:55.102440 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f6d28-23c9-4874-aebd-9252ca325483 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:55.113779 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f6d28-23c9-4874-aebd-9252ca325483 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:55.115449 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61f6d28-23c9-4874-aebd-9252ca325483 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:55.125989 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 938/1855] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:54 2022] POST /identity/v3/projects => generated 373 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:55.125989 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e9cf0-abe1-42f2-96f0-b32c31f39f4f None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:55.126332 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e9cf0-abe1-42f2-96f0-b32c31f39f4f None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:55.126681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e9cf0-abe1-42f2-96f0-b32c31f39f4f None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:55.128197 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 939/1856] 10.222.0.13 () {54 vars in 921 bytes} [Fri May 27 19:50:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:50:55.128978 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e9cf0-abe1-42f2-96f0-b32c31f39f4f None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:55.131055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-2a3e9cf0-abe1-42f2-96f0-b32c31f39f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:55.141934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 918/1857] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:55.143677 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efddda5-ac22-40e9-9c02-90c12146cef3 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:55.144043 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efddda5-ac22-40e9-9c02-90c12146cef3 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:55.144391 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efddda5-ac22-40e9-9c02-90c12146cef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:55.163671 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-5efddda5-ac22-40e9-9c02-90c12146cef3 None None] MFA Rules not processed for user `7542e1fd160e4b2c9d25accf19aad19f`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:55.166029 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-5efddda5-ac22-40e9-9c02-90c12146cef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:55.277828 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e9cf0-abe1-42f2-96f0-b32c31f39f4f None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:55.288871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff988f2-4427-4d77-82ee-c2205e3880c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:55.296674 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff988f2-4427-4d77-82ee-c2205e3880c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:55.296674 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff988f2-4427-4d77-82ee-c2205e3880c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:55.296674 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff988f2-4427-4d77-82ee-c2205e3880c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:55.296674 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff988f2-4427-4d77-82ee-c2205e3880c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:55.296674 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff988f2-4427-4d77-82ee-c2205e3880c0 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:55.307081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 919/1858] 10.222.0.13 () {58 vars in 1018 bytes} [Fri May 27 19:50:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:55.321860 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efddda5-ac22-40e9-9c02-90c12146cef3 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:55.324308 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-5efddda5-ac22-40e9-9c02-90c12146cef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:55.338297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:55.353578 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efddda5-ac22-40e9-9c02-90c12146cef3 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:55.361374 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30d6ea-2be0-474c-97fe-9ec88cb547c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:55.362715 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30d6ea-2be0-474c-97fe-9ec88cb547c5 service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:55.363010 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30d6ea-2be0-474c-97fe-9ec88cb547c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:55.363645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d30d6ea-2be0-474c-97fe-9ec88cb547c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:55.380716 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d30d6ea-2be0-474c-97fe-9ec88cb547c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:55.382381 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d30d6ea-2be0-474c-97fe-9ec88cb547c5 service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:55.401004 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 940/1859] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:55 2022] POST /identity/v3/users => generated 293 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:55.404404 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff988f2-4427-4d77-82ee-c2205e3880c0 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:55.405180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 920/1860] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:55.416808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff988f2-4427-4d77-82ee-c2205e3880c0 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:55.423739 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022cdc23-d008-4f59-8d3c-933a9cfced9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:55.424464 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cdc23-d008-4f59-8d3c-933a9cfced9e admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:55.424770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cdc23-d008-4f59-8d3c-933a9cfced9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:55.425061 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022cdc23-d008-4f59-8d3c-933a9cfced9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:55.427087 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022cdc23-d008-4f59-8d3c-933a9cfced9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:55.428285 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022cdc23-d008-4f59-8d3c-933a9cfced9e admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:55.444231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 941/1861] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:55.449531 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30d6ea-2be0-474c-97fe-9ec88cb547c5 service nova] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:55.463714 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d30d6ea-2be0-474c-97fe-9ec88cb547c5 service nova] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:55.470491 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d717006-6803-4fd2-ad14-4d069ca20d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:55.471812 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d717006-6803-4fd2-ad14-4d069ca20d7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:55.472425 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d717006-6803-4fd2-ad14-4d069ca20d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:55.472871 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d717006-6803-4fd2-ad14-4d069ca20d7d admin admin] PATH_INFO: `/v3/projects/5ab2f42bffd94622bf35b18e160a47c9/users/5de20f1bd36e4137bc3d010357f06b26/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:55.509351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d717006-6803-4fd2-ad14-4d069ca20d7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ab2f42bffd94622bf35b18e160a47c9, user_id=5de20f1bd36e4137bc3d010357f06b26, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:55.513549 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d717006-6803-4fd2-ad14-4d069ca20d7d admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:55.517001 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:55.630114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 921/1862] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:55 2022] PUT /identity/v3/projects/5ab2f42bffd94622bf35b18e160a47c9/users/5de20f1bd36e4137bc3d010357f06b26/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:55.635056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022cdc23-d008-4f59-8d3c-933a9cfced9e admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:55.645370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022cdc23-d008-4f59-8d3c-933a9cfced9e admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:55.710759 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8031eb-c224-4194-9f4e-be4ace3b6ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:55.712397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8031eb-c224-4194-9f4e-be4ace3b6ceb admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:55.712688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8031eb-c224-4194-9f4e-be4ace3b6ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:55.712979 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8031eb-c224-4194-9f4e-be4ace3b6ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:55.713726 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8031eb-c224-4194-9f4e-be4ace3b6ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:55.715451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8031eb-c224-4194-9f4e-be4ace3b6ceb admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:55.729310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 942/1863] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:55 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:55.730203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d717006-6803-4fd2-ad14-4d069ca20d7d admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:55.740504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d717006-6803-4fd2-ad14-4d069ca20d7d admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:55.745431 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f41d61-5a1d-427c-be1e-31529ac04f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:55.746477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f41d61-5a1d-427c-be1e-31529ac04f41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:55.746770 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f41d61-5a1d-427c-be1e-31529ac04f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:55.747046 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f41d61-5a1d-427c-be1e-31529ac04f41 admin admin] PATH_INFO: `/v3/projects/5ab2f42bffd94622bf35b18e160a47c9/users/5de20f1bd36e4137bc3d010357f06b26/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:55.749126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f41d61-5a1d-427c-be1e-31529ac04f41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ab2f42bffd94622bf35b18e160a47c9, user_id=5de20f1bd36e4137bc3d010357f06b26, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:55.751698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f41d61-5a1d-427c-be1e-31529ac04f41 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:55.762229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 922/1864] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:55 2022] PUT /identity/v3/projects/5ab2f42bffd94622bf35b18e160a47c9/users/5de20f1bd36e4137bc3d010357f06b26/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:55.801322 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8031eb-c224-4194-9f4e-be4ace3b6ceb admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:55.803621 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ef8031eb-c224-4194-9f4e-be4ace3b6ceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:55.819078 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f41d61-5a1d-427c-be1e-31529ac04f41 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:55.821407 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-71f41d61-5a1d-427c-be1e-31529ac04f41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:55.832650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8031eb-c224-4194-9f4e-be4ace3b6ceb admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:55.895779 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f41d61-5a1d-427c-be1e-31529ac04f41 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:55.901017 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4b66fd-e0c2-4636-a49a-a7d5004277cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:55.902000 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b66fd-e0c2-4636-a49a-a7d5004277cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:55.902282 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b66fd-e0c2-4636-a49a-a7d5004277cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:55.902554 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4b66fd-e0c2-4636-a49a-a7d5004277cf admin admin] PATH_INFO: `/v3/users/26c84d3f1b8e46bebf02c29a8591adb1` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:55.903773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b66fd-e0c2-4636-a49a-a7d5004277cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=26c84d3f1b8e46bebf02c29a8591adb1)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:55.903875 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624db08-c070-496e-8200-8e04fa221e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:55.904728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624db08-c070-496e-8200-8e04fa221e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:55.905047 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624db08-c070-496e-8200-8e04fa221e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:55.905331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624db08-c070-496e-8200-8e04fa221e12 admin admin] PATH_INFO: `/v3/projects/5bbf9e5e42a949fb892948517a8f7f81` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:55.906203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4b66fd-e0c2-4636-a49a-a7d5004277cf admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:55.907028 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624db08-c070-496e-8200-8e04fa221e12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bbf9e5e42a949fb892948517a8f7f81)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:55.908981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624db08-c070-496e-8200-8e04fa221e12 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:56.191968 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-6c4b66fd-e0c2-4636-a49a-a7d5004277cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26c84d3f1b8e46bebf02c29a8591adb1'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:56.196840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-9624db08-c070-496e-8200-8e04fa221e12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bbf9e5e42a949fb892948517a8f7f81', 'request_id': 'req-9624db08-c070-496e-8200-8e04fa221e12'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:56.206817 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-9624db08-c070-496e-8200-8e04fa221e12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bbf9e5e42a949fb892948517a8f7f81', 'request_id': 'req-9624db08-c070-496e-8200-8e04fa221e12'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:56.211558 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-6c4b66fd-e0c2-4636-a49a-a7d5004277cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26c84d3f1b8e46bebf02c29a8591adb1'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:56.215602 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-9624db08-c070-496e-8200-8e04fa221e12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bbf9e5e42a949fb892948517a8f7f81', 'request_id': 'req-9624db08-c070-496e-8200-8e04fa221e12'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:56.342396 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-6c4b66fd-e0c2-4636-a49a-a7d5004277cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26c84d3f1b8e46bebf02c29a8591adb1'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:56.343056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-6c4b66fd-e0c2-4636-a49a-a7d5004277cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26c84d3f1b8e46bebf02c29a8591adb1'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:56.344131 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-6c4b66fd-e0c2-4636-a49a-a7d5004277cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26c84d3f1b8e46bebf02c29a8591adb1'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:56.358830 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 923/1865] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:50:55 2022] DELETE /identity/v3/users/26c84d3f1b8e46bebf02c29a8591adb1 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:56.363675 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11c4bb-524f-43b6-9320-6e4498e51db8 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:56.363967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11c4bb-524f-43b6-9320-6e4498e51db8 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:56.364254 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11c4bb-524f-43b6-9320-6e4498e51db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:56.389213 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-9d11c4bb-524f-43b6-9320-6e4498e51db8 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:56.389213 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-9d11c4bb-524f-43b6-9320-6e4498e51db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:56.455014 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 924/1866] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:56.456793 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-9624db08-c070-496e-8200-8e04fa221e12 admin admin] The token cache is being invalidate because project 5bbf9e5e42a949fb892948517a8f7f81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:50:56.457354 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-9624db08-c070-496e-8200-8e04fa221e12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:56.458230 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-9624db08-c070-496e-8200-8e04fa221e12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:56.461062 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 943/1867] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:50:55 2022] DELETE /identity/v3/projects/5bbf9e5e42a949fb892948517a8f7f81 => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:56.465699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624db08-c070-496e-8200-8e04fa221e12 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:56.466069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11c4bb-524f-43b6-9320-6e4498e51db8 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:56.467746 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-9624db08-c070-496e-8200-8e04fa221e12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:56.468013 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-9d11c4bb-524f-43b6-9320-6e4498e51db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:56.482422 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11c4bb-524f-43b6-9320-6e4498e51db8 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:56.487401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c87073-1db0-45f1-aa4b-9091b7dd7fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:56.488458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c87073-1db0-45f1-aa4b-9091b7dd7fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:56.488766 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c87073-1db0-45f1-aa4b-9091b7dd7fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:56.489062 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c87073-1db0-45f1-aa4b-9091b7dd7fc7 admin admin] PATH_INFO: `/v3/users/53d3aba4f9c24c80be4277049e342220` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:56.490267 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c87073-1db0-45f1-aa4b-9091b7dd7fc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53d3aba4f9c24c80be4277049e342220)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:56.492186 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c87073-1db0-45f1-aa4b-9091b7dd7fc7 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:56.569370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624db08-c070-496e-8200-8e04fa221e12 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:56.588025 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:56.627216 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824fc76-1113-4e9a-9b9c-2cf2d811d042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:56.627963 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824fc76-1113-4e9a-9b9c-2cf2d811d042 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:56.628285 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824fc76-1113-4e9a-9b9c-2cf2d811d042 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:56.628516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824fc76-1113-4e9a-9b9c-2cf2d811d042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:56.630549 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d824fc76-1113-4e9a-9b9c-2cf2d811d042 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:56.643711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824fc76-1113-4e9a-9b9c-2cf2d811d042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:56.645136 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824fc76-1113-4e9a-9b9c-2cf2d811d042 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:56.656799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 944/1868] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:56.750564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:56.778480 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d4c87073-1db0-45f1-aa4b-9091b7dd7fc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53d3aba4f9c24c80be4277049e342220'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:56.797427 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d4c87073-1db0-45f1-aa4b-9091b7dd7fc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53d3aba4f9c24c80be4277049e342220'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:56.871969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d4c87073-1db0-45f1-aa4b-9091b7dd7fc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53d3aba4f9c24c80be4277049e342220'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:56.873056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d4c87073-1db0-45f1-aa4b-9091b7dd7fc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53d3aba4f9c24c80be4277049e342220'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:56.874041 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d4c87073-1db0-45f1-aa4b-9091b7dd7fc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53d3aba4f9c24c80be4277049e342220'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:56.889943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 925/1869] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:50:56 2022] DELETE /identity/v3/users/53d3aba4f9c24c80be4277049e342220 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:57.217716 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824fc76-1113-4e9a-9b9c-2cf2d811d042 service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:57.219839 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d824fc76-1113-4e9a-9b9c-2cf2d811d042 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:57.234815 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824fc76-1113-4e9a-9b9c-2cf2d811d042 service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:57.308143 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825e3d6-e891-4d51-a50e-15888ac74508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:57.308359 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825e3d6-e891-4d51-a50e-15888ac74508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:57.308666 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825e3d6-e891-4d51-a50e-15888ac74508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:57.309034 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825e3d6-e891-4d51-a50e-15888ac74508 admin admin] PATH_INFO: `/v3/projects/7abae1a562d94cc49e3f9845aa890646` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:57.311590 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825e3d6-e891-4d51-a50e-15888ac74508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7abae1a562d94cc49e3f9845aa890646)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:57.312885 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825e3d6-e891-4d51-a50e-15888ac74508 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:57.469299 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5e09a-018b-4419-83f6-a6e7745454a7 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:57.469661 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5e09a-018b-4419-83f6-a6e7745454a7 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:57.469949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5e09a-018b-4419-83f6-a6e7745454a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:57.504080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c825e3d6-e891-4d51-a50e-15888ac74508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7abae1a562d94cc49e3f9845aa890646', 'request_id': 'req-c825e3d6-e891-4d51-a50e-15888ac74508'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:57.529547 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-7ec5e09a-018b-4419-83f6-a6e7745454a7 None None] MFA Rules not processed for user `5de20f1bd36e4137bc3d010357f06b26`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:57.531210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-7ec5e09a-018b-4419-83f6-a6e7745454a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:57.532312 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c825e3d6-e891-4d51-a50e-15888ac74508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7abae1a562d94cc49e3f9845aa890646', 'request_id': 'req-c825e3d6-e891-4d51-a50e-15888ac74508'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:57.542019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c825e3d6-e891-4d51-a50e-15888ac74508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7abae1a562d94cc49e3f9845aa890646', 'request_id': 'req-c825e3d6-e891-4d51-a50e-15888ac74508'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:57.614415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c825e3d6-e891-4d51-a50e-15888ac74508 admin admin] The token cache is being invalidate because project 7abae1a562d94cc49e3f9845aa890646 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:50:57.614719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c825e3d6-e891-4d51-a50e-15888ac74508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:57.615814 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c825e3d6-e891-4d51-a50e-15888ac74508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:57.622993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 945/1870] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:50:57 2022] DELETE /identity/v3/projects/7abae1a562d94cc49e3f9845aa890646 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:57.624191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825e3d6-e891-4d51-a50e-15888ac74508 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:57.627022 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-c825e3d6-e891-4d51-a50e-15888ac74508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:57.691296 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 926/1871] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:50:57.692262 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5e09a-018b-4419-83f6-a6e7745454a7 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:57.693667 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-7ec5e09a-018b-4419-83f6-a6e7745454a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:57.716746 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825e3d6-e891-4d51-a50e-15888ac74508 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:57.722382 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65573ae4-b3e6-4e8c-91e6-ec85f37c2c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:57.723262 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65573ae4-b3e6-4e8c-91e6-ec85f37c2c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:57.723548 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65573ae4-b3e6-4e8c-91e6-ec85f37c2c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:57.723819 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65573ae4-b3e6-4e8c-91e6-ec85f37c2c2d admin admin] PATH_INFO: `/v3/projects/b78da24e53e24dcf8218e07400955778` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:57.725048 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65573ae4-b3e6-4e8c-91e6-ec85f37c2c2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b78da24e53e24dcf8218e07400955778)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:57.727938 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65573ae4-b3e6-4e8c-91e6-ec85f37c2c2d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:57.729284 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:57.769986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5e09a-018b-4419-83f6-a6e7745454a7 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:57.775397 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db84475-826c-49ae-af78-bb284e7a4fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:57.776287 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db84475-826c-49ae-af78-bb284e7a4fb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:57.776577 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db84475-826c-49ae-af78-bb284e7a4fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:57.776862 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db84475-826c-49ae-af78-bb284e7a4fb4 admin admin] PATH_INFO: `/v3/users/70fb17b1774f4b738b45411c27dfd0e2` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:57.778148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db84475-826c-49ae-af78-bb284e7a4fb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70fb17b1774f4b738b45411c27dfd0e2)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:57.780088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db84475-826c-49ae-af78-bb284e7a4fb4 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:57.839809 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-65573ae4-b3e6-4e8c-91e6-ec85f37c2c2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b78da24e53e24dcf8218e07400955778', 'request_id': 'req-65573ae4-b3e6-4e8c-91e6-ec85f37c2c2d'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:57.846504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-65573ae4-b3e6-4e8c-91e6-ec85f37c2c2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b78da24e53e24dcf8218e07400955778', 'request_id': 'req-65573ae4-b3e6-4e8c-91e6-ec85f37c2c2d'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:57.855231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-65573ae4-b3e6-4e8c-91e6-ec85f37c2c2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b78da24e53e24dcf8218e07400955778', 'request_id': 'req-65573ae4-b3e6-4e8c-91e6-ec85f37c2c2d'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:57.941745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:57.948766 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-65573ae4-b3e6-4e8c-91e6-ec85f37c2c2d admin admin] The token cache is being invalidate because project b78da24e53e24dcf8218e07400955778 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:50:57.949084 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-65573ae4-b3e6-4e8c-91e6-ec85f37c2c2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:57.949790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-65573ae4-b3e6-4e8c-91e6-ec85f37c2c2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:57.954973 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 946/1872] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:50:57 2022] DELETE /identity/v3/projects/b78da24e53e24dcf8218e07400955778 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:57.954973 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65573ae4-b3e6-4e8c-91e6-ec85f37c2c2d admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:57.955821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-65573ae4-b3e6-4e8c-91e6-ec85f37c2c2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:58.027097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65573ae4-b3e6-4e8c-91e6-ec85f37c2c2d admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:58.030831 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d910a5-8ead-444d-8f4c-964f51f40188 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:58.031481 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d910a5-8ead-444d-8f4c-964f51f40188 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:58.031766 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d910a5-8ead-444d-8f4c-964f51f40188 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:58.032047 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d910a5-8ead-444d-8f4c-964f51f40188 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:58.036643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-78d910a5-8ead-444d-8f4c-964f51f40188 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:58.069761 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-2db84475-826c-49ae-af78-bb284e7a4fb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70fb17b1774f4b738b45411c27dfd0e2'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:58.082481 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-2db84475-826c-49ae-af78-bb284e7a4fb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70fb17b1774f4b738b45411c27dfd0e2'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:58.094064 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d910a5-8ead-444d-8f4c-964f51f40188 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:58.095517 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d910a5-8ead-444d-8f4c-964f51f40188 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:58.104403 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 947/1873] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:50:58.105704 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0c065-3162-4c53-a05b-721224bbace3 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:58.106015 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0c065-3162-4c53-a05b-721224bbace3 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:58.106306 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0c065-3162-4c53-a05b-721224bbace3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:58.124236 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-1da0c065-3162-4c53-a05b-721224bbace3 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:58.128508 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-1da0c065-3162-4c53-a05b-721224bbace3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:58.170450 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-2db84475-826c-49ae-af78-bb284e7a4fb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70fb17b1774f4b738b45411c27dfd0e2'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:58.171625 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-2db84475-826c-49ae-af78-bb284e7a4fb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70fb17b1774f4b738b45411c27dfd0e2'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:58.173101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-2db84475-826c-49ae-af78-bb284e7a4fb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70fb17b1774f4b738b45411c27dfd0e2'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:58.187074 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 948/1874] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:58.187802 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 927/1875] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:50:57 2022] DELETE /identity/v3/users/70fb17b1774f4b738b45411c27dfd0e2 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:58.196045 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f0b49-d670-4e7b-bc66-f94a28c44cde None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:58.196350 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f0b49-d670-4e7b-bc66-f94a28c44cde None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:58.196639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4f0b49-d670-4e7b-bc66-f94a28c44cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:58.217954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-3f4f0b49-d670-4e7b-bc66-f94a28c44cde None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:58.219547 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-3f4f0b49-d670-4e7b-bc66-f94a28c44cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:58.285797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 949/1876] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:50:58.290271 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db84475-826c-49ae-af78-bb284e7a4fb4 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:58.301515 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db84475-826c-49ae-af78-bb284e7a4fb4 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:58.306971 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4de61f-42f2-4ef0-9660-3fc90044c8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:58.307640 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4de61f-42f2-4ef0-9660-3fc90044c8dc admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:58.307924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4de61f-42f2-4ef0-9660-3fc90044c8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:58.308205 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4de61f-42f2-4ef0-9660-3fc90044c8dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:58.308972 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4de61f-42f2-4ef0-9660-3fc90044c8dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:50:58.309284 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4de61f-42f2-4ef0-9660-3fc90044c8dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:58.310899 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4de61f-42f2-4ef0-9660-3fc90044c8dc admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:58.323495 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 928/1877] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:50:58.328105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f0b49-d670-4e7b-bc66-f94a28c44cde None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:58.344311 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4f0b49-d670-4e7b-bc66-f94a28c44cde None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:58.346280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc0648-d43b-446b-876d-5aed5342aedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:58.347304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc0648-d43b-446b-876d-5aed5342aedf admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:58.347781 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc0648-d43b-446b-876d-5aed5342aedf admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:58.348259 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adc0648-d43b-446b-876d-5aed5342aedf admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:58.349370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc0648-d43b-446b-876d-5aed5342aedf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:58.352856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adc0648-d43b-446b-876d-5aed5342aedf admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:58.471315 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 950/1878] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:58 2022] POST /identity/v3/projects => generated 355 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:58.472818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4de61f-42f2-4ef0-9660-3fc90044c8dc admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:58.484035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4de61f-42f2-4ef0-9660-3fc90044c8dc admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:58.551812 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be094740-37ec-43c0-8cb7-56628763952f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:58.552492 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be094740-37ec-43c0-8cb7-56628763952f admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:58.552778 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be094740-37ec-43c0-8cb7-56628763952f admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:58.553091 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be094740-37ec-43c0-8cb7-56628763952f admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:58.553821 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be094740-37ec-43c0-8cb7-56628763952f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:58.557073 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be094740-37ec-43c0-8cb7-56628763952f admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:58.630222 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc0648-d43b-446b-876d-5aed5342aedf admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:58.632463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-8adc0648-d43b-446b-876d-5aed5342aedf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:58.646802 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adc0648-d43b-446b-876d-5aed5342aedf admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:58.654403 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a468adc-4a73-4e42-b174-dd240cf2554f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:58.654403 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a468adc-4a73-4e42-b174-dd240cf2554f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:58.654403 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a468adc-4a73-4e42-b174-dd240cf2554f admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:58.654403 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a468adc-4a73-4e42-b174-dd240cf2554f admin admin] PATH_INFO: `/v3/projects/c5bc5d4177f74942ae15db1629dbb7d5` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:58.656795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a468adc-4a73-4e42-b174-dd240cf2554f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5bc5d4177f74942ae15db1629dbb7d5)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:58.659002 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a468adc-4a73-4e42-b174-dd240cf2554f admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:58.680238 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 929/1879] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:58 2022] POST /identity/v3/users => generated 278 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:58.684812 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be094740-37ec-43c0-8cb7-56628763952f admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:58.696048 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be094740-37ec-43c0-8cb7-56628763952f admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:58.720691 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2781271-2912-4c64-8523-c8c9aadebd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:58.722499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2781271-2912-4c64-8523-c8c9aadebd77 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:58.722788 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2781271-2912-4c64-8523-c8c9aadebd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:58.723074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2781271-2912-4c64-8523-c8c9aadebd77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:58.726225 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2781271-2912-4c64-8523-c8c9aadebd77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:58.726225 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2781271-2912-4c64-8523-c8c9aadebd77 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:58.744591 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 930/1880] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:50:58.748715 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2781271-2912-4c64-8523-c8c9aadebd77 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:58.760116 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2781271-2912-4c64-8523-c8c9aadebd77 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:58.766172 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f20f95a-0fba-4e1e-b47a-875e4cf7d776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:58.766865 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20f95a-0fba-4e1e-b47a-875e4cf7d776 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:58.767224 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20f95a-0fba-4e1e-b47a-875e4cf7d776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:58.767575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f20f95a-0fba-4e1e-b47a-875e4cf7d776 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:58.768545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f20f95a-0fba-4e1e-b47a-875e4cf7d776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:58.770672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f20f95a-0fba-4e1e-b47a-875e4cf7d776 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:58.789192 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 931/1881] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:58.790287 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f20f95a-0fba-4e1e-b47a-875e4cf7d776 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:58.802028 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-7a468adc-4a73-4e42-b174-dd240cf2554f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5bc5d4177f74942ae15db1629dbb7d5', 'request_id': 'req-7a468adc-4a73-4e42-b174-dd240cf2554f'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:58.803305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f20f95a-0fba-4e1e-b47a-875e4cf7d776 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:58.808391 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-7a468adc-4a73-4e42-b174-dd240cf2554f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5bc5d4177f74942ae15db1629dbb7d5', 'request_id': 'req-7a468adc-4a73-4e42-b174-dd240cf2554f'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:58.815475 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-7a468adc-4a73-4e42-b174-dd240cf2554f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5bc5d4177f74942ae15db1629dbb7d5', 'request_id': 'req-7a468adc-4a73-4e42-b174-dd240cf2554f'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:58.869473 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a13589-91bf-4a59-a484-850eafb2781d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:58.870376 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a13589-91bf-4a59-a484-850eafb2781d admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:58.870668 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a13589-91bf-4a59-a484-850eafb2781d admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:58.870945 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a13589-91bf-4a59-a484-850eafb2781d admin admin] PATH_INFO: `/v3/projects/3ae8fafc3b0842859e669dd38fda108b/users/181f4390891b478a840be24f469cf468/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:58.886111 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a13589-91bf-4a59-a484-850eafb2781d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ae8fafc3b0842859e669dd38fda108b, user_id=181f4390891b478a840be24f469cf468, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:58.888759 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a13589-91bf-4a59-a484-850eafb2781d admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:58.912550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-7a468adc-4a73-4e42-b174-dd240cf2554f admin admin] The token cache is being invalidate because project c5bc5d4177f74942ae15db1629dbb7d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:50:58.913115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-7a468adc-4a73-4e42-b174-dd240cf2554f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:58.913499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-7a468adc-4a73-4e42-b174-dd240cf2554f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:50:58.915107 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 951/1882] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:50:58 2022] DELETE /identity/v3/projects/c5bc5d4177f74942ae15db1629dbb7d5 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:58.919903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29234018-f6f6-4276-b9a1-60b8a4a94ac2 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:58.920194 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29234018-f6f6-4276-b9a1-60b8a4a94ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:58.920476 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29234018-f6f6-4276-b9a1-60b8a4a94ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:58.938961 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-29234018-f6f6-4276-b9a1-60b8a4a94ac2 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:58.940162 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-29234018-f6f6-4276-b9a1-60b8a4a94ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:58.950331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 952/1883] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:50:58.958985 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd73f6-6f04-44b9-8936-779e20fa53c6 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:58.959372 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd73f6-6f04-44b9-8936-779e20fa53c6 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:58.959654 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd73f6-6f04-44b9-8936-779e20fa53c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:58.978325 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-c8bd73f6-6f04-44b9-8936-779e20fa53c6 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:58.979506 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-c8bd73f6-6f04-44b9-8936-779e20fa53c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:58.987368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 953/1884] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:50:58.991414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd73f6-6f04-44b9-8936-779e20fa53c6 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:59.000692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd73f6-6f04-44b9-8936-779e20fa53c6 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:59.006112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5c7c0-a74e-45d5-a16b-eed1bf7051ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:59.008716 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5c7c0-a74e-45d5-a16b-eed1bf7051ef admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:59.008716 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5c7c0-a74e-45d5-a16b-eed1bf7051ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:59.008716 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5c7c0-a74e-45d5-a16b-eed1bf7051ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:59.008716 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5c7c0-a74e-45d5-a16b-eed1bf7051ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:50:59.009123 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5c7c0-a74e-45d5-a16b-eed1bf7051ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:59.011120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5c7c0-a74e-45d5-a16b-eed1bf7051ef admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:59.011661 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 932/1885] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:58 2022] PUT /identity/v3/projects/3ae8fafc3b0842859e669dd38fda108b/users/181f4390891b478a840be24f469cf468/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:59.023378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471e493-739b-4cc3-87d9-00677a52d840 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:59.023677 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471e493-739b-4cc3-87d9-00677a52d840 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:59.024110 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5471e493-739b-4cc3-87d9-00677a52d840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:59.027354 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 954/1886] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:50:59.028232 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5c7c0-a74e-45d5-a16b-eed1bf7051ef admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:59.038591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5c7c0-a74e-45d5-a16b-eed1bf7051ef admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:59.054334 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-5471e493-739b-4cc3-87d9-00677a52d840 None None] MFA Rules not processed for user `181f4390891b478a840be24f469cf468`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:59.057238 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-5471e493-739b-4cc3-87d9-00677a52d840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:59.106834 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c179be5-f218-440a-8298-27e7ca99ef01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:59.108066 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c179be5-f218-440a-8298-27e7ca99ef01 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:59.109142 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c179be5-f218-440a-8298-27e7ca99ef01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:59.109435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c179be5-f218-440a-8298-27e7ca99ef01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:59.110303 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c179be5-f218-440a-8298-27e7ca99ef01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:59.112798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c179be5-f218-440a-8298-27e7ca99ef01 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:59.152403 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 933/1887] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:59 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:50:59.154429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471e493-739b-4cc3-87d9-00677a52d840 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:59.155872 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-5471e493-739b-4cc3-87d9-00677a52d840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:59.170853 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5471e493-739b-4cc3-87d9-00677a52d840 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:59.176153 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:59.179697 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e335f0-79db-4752-9be1-1f0097196418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:59.181006 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e335f0-79db-4752-9be1-1f0097196418 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:59.181335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e335f0-79db-4752-9be1-1f0097196418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:59.181625 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e335f0-79db-4752-9be1-1f0097196418 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:59.182339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e335f0-79db-4752-9be1-1f0097196418 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:59.184494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e335f0-79db-4752-9be1-1f0097196418 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:59.230522 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 955/1888] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:59 2022] POST /identity/v3/projects => generated 371 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:59.235090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c179be5-f218-440a-8298-27e7ca99ef01 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:59.282898 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 934/1889] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:50:59 2022] POST /identity/v3/projects => generated 355 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:59.287350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e335f0-79db-4752-9be1-1f0097196418 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:59.307449 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:59.311246 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c179be5-f218-440a-8298-27e7ca99ef01 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:59.368197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e335f0-79db-4752-9be1-1f0097196418 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:59.380453 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990fcd0f-4ec1-48c5-8e9b-4531484a1d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:59.381201 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fcd0f-4ec1-48c5-8e9b-4531484a1d51 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:59.381491 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fcd0f-4ec1-48c5-8e9b-4531484a1d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:59.381773 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fcd0f-4ec1-48c5-8e9b-4531484a1d51 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:59.381962 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5e3fd-0803-4343-af0c-0f57852cad5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:59.382470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990fcd0f-4ec1-48c5-8e9b-4531484a1d51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:59.382658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5e3fd-0803-4343-af0c-0f57852cad5c admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:59.383497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5e3fd-0803-4343-af0c-0f57852cad5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:59.383982 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5e3fd-0803-4343-af0c-0f57852cad5c admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:59.384699 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b5e3fd-0803-4343-af0c-0f57852cad5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:59.385142 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990fcd0f-4ec1-48c5-8e9b-4531484a1d51 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:59.386760 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b5e3fd-0803-4343-af0c-0f57852cad5c admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:59.441911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 956/1890] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:59 2022] POST /identity/v3/users => generated 286 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:50:59.442091 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 935/1891] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:50:59 2022] POST /identity/v3/users => generated 284 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:50:59.443346 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990fcd0f-4ec1-48c5-8e9b-4531484a1d51 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:59.445220 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5e3fd-0803-4343-af0c-0f57852cad5c admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:59.453259 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990fcd0f-4ec1-48c5-8e9b-4531484a1d51 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:59.455125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b5e3fd-0803-4343-af0c-0f57852cad5c admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:59.458108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfcd8e-7f91-4b38-bee4-a39a327a50a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:59.458770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfcd8e-7f91-4b38-bee4-a39a327a50a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:59.459058 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfcd8e-7f91-4b38-bee4-a39a327a50a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:59.459336 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfcd8e-7f91-4b38-bee4-a39a327a50a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:59.459819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ccadb-e5fa-4776-ba3e-47cfffc8df87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:59.460061 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfcd8e-7f91-4b38-bee4-a39a327a50a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:59.460480 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ccadb-e5fa-4776-ba3e-47cfffc8df87 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:59.460771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ccadb-e5fa-4776-ba3e-47cfffc8df87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:59.461128 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ccadb-e5fa-4776-ba3e-47cfffc8df87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:59.461734 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfcd8e-7f91-4b38-bee4-a39a327a50a1 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:59.462186 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ccadb-e5fa-4776-ba3e-47cfffc8df87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:59.463775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ccadb-e5fa-4776-ba3e-47cfffc8df87 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:59.472681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 957/1892] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:50:59.475047 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 936/1893] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:59.477450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfcd8e-7f91-4b38-bee4-a39a327a50a1 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:59.478619 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ccadb-e5fa-4776-ba3e-47cfffc8df87 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:59.487761 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfcd8e-7f91-4b38-bee4-a39a327a50a1 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:59.488722 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ccadb-e5fa-4776-ba3e-47cfffc8df87 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:59.493266 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e607d-5873-4f4c-93ae-04c94e91f655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:59.493937 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e607d-5873-4f4c-93ae-04c94e91f655 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:59.494174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f9137-6218-4bef-baa4-e8704f001675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:59.494300 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e607d-5873-4f4c-93ae-04c94e91f655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:59.494479 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e607d-5873-4f4c-93ae-04c94e91f655 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:59.495057 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f9137-6218-4bef-baa4-e8704f001675 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:59.495243 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e607d-5873-4f4c-93ae-04c94e91f655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:59.495332 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f9137-6218-4bef-baa4-e8704f001675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:59.495624 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f9137-6218-4bef-baa4-e8704f001675 admin admin] PATH_INFO: `/v3/projects/3c802b45ef5f46af85aa28ddd56eee70/users/a80533b0d89e49a3932ce6dec7dfe060/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:59.496853 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e607d-5873-4f4c-93ae-04c94e91f655 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:59.511451 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f9137-6218-4bef-baa4-e8704f001675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c802b45ef5f46af85aa28ddd56eee70, user_id=a80533b0d89e49a3932ce6dec7dfe060, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:59.515318 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 958/1894] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:50:59.516034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6f9137-6218-4bef-baa4-e8704f001675 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:59.516142 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e607d-5873-4f4c-93ae-04c94e91f655 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:59.526134 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e607d-5873-4f4c-93ae-04c94e91f655 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:59.531036 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea13de-46f3-4c45-b487-842c00fe1b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:59.531959 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea13de-46f3-4c45-b487-842c00fe1b5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:59.532244 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea13de-46f3-4c45-b487-842c00fe1b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:59.532518 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea13de-46f3-4c45-b487-842c00fe1b5f admin admin] PATH_INFO: `/v3/projects/c40928a55f3d4dcb82bbb3c4244f7f93/users/239d7c2d311c48728d3237ef580290e0/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:59.549028 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea13de-46f3-4c45-b487-842c00fe1b5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c40928a55f3d4dcb82bbb3c4244f7f93, user_id=239d7c2d311c48728d3237ef580290e0, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:59.551852 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea13de-46f3-4c45-b487-842c00fe1b5f admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:59.580421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 937/1895] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:59 2022] PUT /identity/v3/projects/3c802b45ef5f46af85aa28ddd56eee70/users/a80533b0d89e49a3932ce6dec7dfe060/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:50:59.584212 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f9137-6218-4bef-baa4-e8704f001675 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:59.644593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f9137-6218-4bef-baa4-e8704f001675 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:59.653225 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716ffa8-bffd-4ca6-bf4c-0586aa0b5eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:59.653225 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716ffa8-bffd-4ca6-bf4c-0586aa0b5eca admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:59.653225 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716ffa8-bffd-4ca6-bf4c-0586aa0b5eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:59.653225 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716ffa8-bffd-4ca6-bf4c-0586aa0b5eca admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:59.653746 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716ffa8-bffd-4ca6-bf4c-0586aa0b5eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:59.653746 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716ffa8-bffd-4ca6-bf4c-0586aa0b5eca admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:59.666082 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 959/1896] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:59 2022] PUT /identity/v3/projects/c40928a55f3d4dcb82bbb3c4244f7f93/users/239d7c2d311c48728d3237ef580290e0/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:50:59.666847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 938/1897] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:50:59.667730 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea13de-46f3-4c45-b487-842c00fe1b5f admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:59.668525 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2186f6-1318-4e8b-bb9f-623994790598 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:59.668818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2186f6-1318-4e8b-bb9f-623994790598 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:59.669131 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2186f6-1318-4e8b-bb9f-623994790598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:59.698057 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-ff2186f6-1318-4e8b-bb9f-623994790598 None None] MFA Rules not processed for user `239d7c2d311c48728d3237ef580290e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:59.699263 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-ff2186f6-1318-4e8b-bb9f-623994790598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:59.732137 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea13de-46f3-4c45-b487-842c00fe1b5f admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:59.737254 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aaa8a3-87d0-4dd1-ab25-b577a6a47b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:59.738993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aaa8a3-87d0-4dd1-ab25-b577a6a47b0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:59.738993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aaa8a3-87d0-4dd1-ab25-b577a6a47b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:59.738993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aaa8a3-87d0-4dd1-ab25-b577a6a47b0f admin admin] PATH_INFO: `/v3/projects/3c802b45ef5f46af85aa28ddd56eee70/users/a80533b0d89e49a3932ce6dec7dfe060/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:59.741155 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aaa8a3-87d0-4dd1-ab25-b577a6a47b0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c802b45ef5f46af85aa28ddd56eee70, user_id=a80533b0d89e49a3932ce6dec7dfe060, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:59.744468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aaa8a3-87d0-4dd1-ab25-b577a6a47b0f admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:59.756393 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 960/1898] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:50:59 2022] PUT /identity/v3/projects/3c802b45ef5f46af85aa28ddd56eee70/users/a80533b0d89e49a3932ce6dec7dfe060/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:50:59.765170 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 939/1899] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:50:59.767194 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf30ecc-e152-497d-b864-a0835b6abbfd None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:59.767491 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf30ecc-e152-497d-b864-a0835b6abbfd None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:59.770211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf30ecc-e152-497d-b864-a0835b6abbfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:59.778460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2186f6-1318-4e8b-bb9f-623994790598 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:59.780013 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-ff2186f6-1318-4e8b-bb9f-623994790598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:59.799402 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-3bf30ecc-e152-497d-b864-a0835b6abbfd None None] MFA Rules not processed for user `a80533b0d89e49a3932ce6dec7dfe060`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:50:59.801079 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-3bf30ecc-e152-497d-b864-a0835b6abbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:59.845553 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:50:59.846949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2186f6-1318-4e8b-bb9f-623994790598 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:59.879094 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 961/1900] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:50:59 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:50:59.893364 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf30ecc-e152-497d-b864-a0835b6abbfd None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:50:59.894995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-3bf30ecc-e152-497d-b864-a0835b6abbfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:50:59.904087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1079c52-4284-4d2f-8c2c-85d80853b6cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:59.904837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1079c52-4284-4d2f-8c2c-85d80853b6cf service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:59.905224 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1079c52-4284-4d2f-8c2c-85d80853b6cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:59.905576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1079c52-4284-4d2f-8c2c-85d80853b6cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:59.962057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf30ecc-e152-497d-b864-a0835b6abbfd None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:50:59.966802 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bceb44b-36f6-4e73-a299-62347d64c684 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:50:59.967467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bceb44b-36f6-4e73-a299-62347d64c684 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:50:59.967764 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bceb44b-36f6-4e73-a299-62347d64c684 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:50:59.968047 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bceb44b-36f6-4e73-a299-62347d64c684 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:50:59.971828 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1079c52-4284-4d2f-8c2c-85d80853b6cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:59.973230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1079c52-4284-4d2f-8c2c-85d80853b6cf service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:59.981113 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bceb44b-36f6-4e73-a299-62347d64c684 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:50:59.982718 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bceb44b-36f6-4e73-a299-62347d64c684 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:50:59.984447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 940/1901] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:51:00.000711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 962/1902] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:50:59 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:51:00.005189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:00.007599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1079c52-4284-4d2f-8c2c-85d80853b6cf service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:00.067633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1079c52-4284-4d2f-8c2c-85d80853b6cf service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:00.072301 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c0551-9882-418e-a3ef-3aa233b099a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:00.073193 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c0551-9882-418e-a3ef-3aa233b099a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:00.073477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c0551-9882-418e-a3ef-3aa233b099a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:00.073985 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14c0551-9882-418e-a3ef-3aa233b099a4 admin admin] PATH_INFO: `/v3/users/239d7c2d311c48728d3237ef580290e0` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:00.074939 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14c0551-9882-418e-a3ef-3aa233b099a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=239d7c2d311c48728d3237ef580290e0)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:00.076838 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14c0551-9882-418e-a3ef-3aa233b099a4 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:00.222517 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bceb44b-36f6-4e73-a299-62347d64c684 service cinder] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:00.233694 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bceb44b-36f6-4e73-a299-62347d64c684 service cinder] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:00.235516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50378114-55aa-485d-8c60-f0827abe05b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:00.236161 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50378114-55aa-485d-8c60-f0827abe05b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:00.236451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50378114-55aa-485d-8c60-f0827abe05b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:00.236762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50378114-55aa-485d-8c60-f0827abe05b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:00.263771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f14c0551-9882-418e-a3ef-3aa233b099a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '239d7c2d311c48728d3237ef580290e0'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:00.280018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f14c0551-9882-418e-a3ef-3aa233b099a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '239d7c2d311c48728d3237ef580290e0'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:00.298207 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50378114-55aa-485d-8c60-f0827abe05b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:00.300746 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50378114-55aa-485d-8c60-f0827abe05b4 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:00.314874 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 963/1903] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:00 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:51:00.354007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f14c0551-9882-418e-a3ef-3aa233b099a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '239d7c2d311c48728d3237ef580290e0'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:00.354673 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f14c0551-9882-418e-a3ef-3aa233b099a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '239d7c2d311c48728d3237ef580290e0'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:00.355543 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f14c0551-9882-418e-a3ef-3aa233b099a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '239d7c2d311c48728d3237ef580290e0'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:00.368680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 941/1904] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:00 2022] DELETE /identity/v3/users/239d7c2d311c48728d3237ef580290e0 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:00.373269 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f81cc-f7d0-4543-95fd-bcf72dc5b6e4 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:00.373572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f81cc-f7d0-4543-95fd-bcf72dc5b6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:00.373856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f81cc-f7d0-4543-95fd-bcf72dc5b6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:00.396672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-637f81cc-f7d0-4543-95fd-bcf72dc5b6e4 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:00.397977 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-637f81cc-f7d0-4543-95fd-bcf72dc5b6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:00.439145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:00.462380 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 964/1905] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:00.472065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c0551-9882-418e-a3ef-3aa233b099a4 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:00.473611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-f14c0551-9882-418e-a3ef-3aa233b099a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:00.531894 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:DELETE=6,keystone:SELECT=94 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:00.538367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14c0551-9882-418e-a3ef-3aa233b099a4 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:00.543067 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d92896-ce88-4e32-8694-50536aebe1e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:00.543742 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d92896-ce88-4e32-8694-50536aebe1e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:00.544056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d92896-ce88-4e32-8694-50536aebe1e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:00.544320 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d92896-ce88-4e32-8694-50536aebe1e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:00.553834 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d92896-ce88-4e32-8694-50536aebe1e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:00.555592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d92896-ce88-4e32-8694-50536aebe1e3 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:00.565125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 942/1906] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:00.949465 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f81cc-f7d0-4543-95fd-bcf72dc5b6e4 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:00.950885 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-637f81cc-f7d0-4543-95fd-bcf72dc5b6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:00.970208 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f81cc-f7d0-4543-95fd-bcf72dc5b6e4 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:00.971382 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7dd47-3a42-43c6-91ad-015ee51abc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:00.972753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7dd47-3a42-43c6-91ad-015ee51abc2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:00.973232 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7dd47-3a42-43c6-91ad-015ee51abc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:00.973527 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7dd47-3a42-43c6-91ad-015ee51abc2d admin admin] PATH_INFO: `/v3/projects/c40928a55f3d4dcb82bbb3c4244f7f93` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:00.974826 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7dd47-3a42-43c6-91ad-015ee51abc2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c40928a55f3d4dcb82bbb3c4244f7f93)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:00.976781 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7dd47-3a42-43c6-91ad-015ee51abc2d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:01.033780 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:01.107655 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-dec7dd47-3a42-43c6-91ad-015ee51abc2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c40928a55f3d4dcb82bbb3c4244f7f93', 'request_id': 'req-dec7dd47-3a42-43c6-91ad-015ee51abc2d'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:01.113265 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-dec7dd47-3a42-43c6-91ad-015ee51abc2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c40928a55f3d4dcb82bbb3c4244f7f93', 'request_id': 'req-dec7dd47-3a42-43c6-91ad-015ee51abc2d'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:01.119940 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-dec7dd47-3a42-43c6-91ad-015ee51abc2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c40928a55f3d4dcb82bbb3c4244f7f93', 'request_id': 'req-dec7dd47-3a42-43c6-91ad-015ee51abc2d'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:01.191123 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-dec7dd47-3a42-43c6-91ad-015ee51abc2d admin admin] The token cache is being invalidate because project c40928a55f3d4dcb82bbb3c4244f7f93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:51:01.191407 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-dec7dd47-3a42-43c6-91ad-015ee51abc2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:01.192131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-dec7dd47-3a42-43c6-91ad-015ee51abc2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:01.193635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 965/1907] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:00 2022] DELETE /identity/v3/projects/c40928a55f3d4dcb82bbb3c4244f7f93 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:01.199026 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106607c-73a6-4bd8-b019-084c7418d9b8 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:01.199484 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106607c-73a6-4bd8-b019-084c7418d9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:01.199589 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106607c-73a6-4bd8-b019-084c7418d9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:01.204534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:01.220118 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-0106607c-73a6-4bd8-b019-084c7418d9b8 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:01.221475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-0106607c-73a6-4bd8-b019-084c7418d9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:01.281576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 943/1908] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:01.290606 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04edb1ed-35c3-4ff0-b211-ef0ffead083d None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:01.290901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04edb1ed-35c3-4ff0-b211-ef0ffead083d None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:01.291186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04edb1ed-35c3-4ff0-b211-ef0ffead083d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:01.309699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-04edb1ed-35c3-4ff0-b211-ef0ffead083d None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:01.310894 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-04edb1ed-35c3-4ff0-b211-ef0ffead083d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:01.318755 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 966/1909] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:01.324342 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106607c-73a6-4bd8-b019-084c7418d9b8 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:01.333967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106607c-73a6-4bd8-b019-084c7418d9b8 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:01.338801 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6410a83-97ca-4f38-be4b-560712414f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:01.339462 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6410a83-97ca-4f38-be4b-560712414f5b admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:01.339749 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6410a83-97ca-4f38-be4b-560712414f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:01.340033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6410a83-97ca-4f38-be4b-560712414f5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:01.340765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6410a83-97ca-4f38-be4b-560712414f5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:51:01.341110 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6410a83-97ca-4f38-be4b-560712414f5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:01.342713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6410a83-97ca-4f38-be4b-560712414f5b admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:01.358698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 944/1910] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:51:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:51:01.363092 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04edb1ed-35c3-4ff0-b211-ef0ffead083d None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:01.372840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04edb1ed-35c3-4ff0-b211-ef0ffead083d None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:01.378948 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936b486-2488-42cb-9590-d0cbfdf169e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:01.379601 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936b486-2488-42cb-9590-d0cbfdf169e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:01.379885 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936b486-2488-42cb-9590-d0cbfdf169e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:01.380161 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936b486-2488-42cb-9590-d0cbfdf169e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:01.380859 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4936b486-2488-42cb-9590-d0cbfdf169e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:01.382801 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4936b486-2488-42cb-9590-d0cbfdf169e3 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:01.460703 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 967/1911] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:51:01 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:01.462028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6410a83-97ca-4f38-be4b-560712414f5b admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:01.472532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6410a83-97ca-4f38-be4b-560712414f5b admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:01.532275 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa3e65-b1e1-44ab-9e5c-fad516f17b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:01.533116 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa3e65-b1e1-44ab-9e5c-fad516f17b1d admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:01.533403 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa3e65-b1e1-44ab-9e5c-fad516f17b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:01.533682 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa3e65-b1e1-44ab-9e5c-fad516f17b1d admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:01.534549 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa3e65-b1e1-44ab-9e5c-fad516f17b1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:01.536496 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa3e65-b1e1-44ab-9e5c-fad516f17b1d admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:01.640003 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 945/1912] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:51:01 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:01.644254 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936b486-2488-42cb-9590-d0cbfdf169e3 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:01.654822 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936b486-2488-42cb-9590-d0cbfdf169e3 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:01.659479 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2399a-d7d7-4764-82a3-40c39b66a492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:01.660440 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2399a-d7d7-4764-82a3-40c39b66a492 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:01.661030 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2399a-d7d7-4764-82a3-40c39b66a492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:01.661725 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2399a-d7d7-4764-82a3-40c39b66a492 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:01.662884 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2399a-d7d7-4764-82a3-40c39b66a492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:01.664625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2399a-d7d7-4764-82a3-40c39b66a492 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:01.677656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 968/1913] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:01.686163 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa3e65-b1e1-44ab-9e5c-fad516f17b1d admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:01.697528 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa3e65-b1e1-44ab-9e5c-fad516f17b1d admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:01.707127 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c4740-876b-4ff1-88cb-7fac39e12281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:01.707127 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c4740-876b-4ff1-88cb-7fac39e12281 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:01.707127 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c4740-876b-4ff1-88cb-7fac39e12281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:01.707127 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c4740-876b-4ff1-88cb-7fac39e12281 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:01.707434 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c4740-876b-4ff1-88cb-7fac39e12281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:01.707434 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c4740-876b-4ff1-88cb-7fac39e12281 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:01.722116 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 946/1914] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:01.723392 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2399a-d7d7-4764-82a3-40c39b66a492 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:01.733943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2399a-d7d7-4764-82a3-40c39b66a492 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:01.738556 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011db1e-969a-43e3-9a10-7b4b146fcbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:01.739435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011db1e-969a-43e3-9a10-7b4b146fcbf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:01.739716 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011db1e-969a-43e3-9a10-7b4b146fcbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:01.740058 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011db1e-969a-43e3-9a10-7b4b146fcbf2 admin admin] PATH_INFO: `/v3/projects/081822f237724bdfab99001aebdb6452/users/aaa08c8b3528410e931f6831bf6ed5d0/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:01.744175 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:01.758225 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6011db1e-969a-43e3-9a10-7b4b146fcbf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=081822f237724bdfab99001aebdb6452, user_id=aaa08c8b3528410e931f6831bf6ed5d0, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:01.761081 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6011db1e-969a-43e3-9a10-7b4b146fcbf2 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:01.810495 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:01.862163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 969/1915] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:01 2022] PUT /identity/v3/projects/081822f237724bdfab99001aebdb6452/users/aaa08c8b3528410e931f6831bf6ed5d0/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:01.873090 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbce6a7-aba3-4181-a2c1-c7dd91b3cd8f None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:01.873388 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbce6a7-aba3-4181-a2c1-c7dd91b3cd8f None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:01.873669 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbce6a7-aba3-4181-a2c1-c7dd91b3cd8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:01.900624 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-ecbce6a7-aba3-4181-a2c1-c7dd91b3cd8f None None] MFA Rules not processed for user `aaa08c8b3528410e931f6831bf6ed5d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:01.901862 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-ecbce6a7-aba3-4181-a2c1-c7dd91b3cd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:01.966082 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 947/1916] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:01.980276 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011db1e-969a-43e3-9a10-7b4b146fcbf2 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:01.980276 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-6011db1e-969a-43e3-9a10-7b4b146fcbf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:02.035702 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011db1e-969a-43e3-9a10-7b4b146fcbf2 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:02.039591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db57440e-a1cf-4fed-953d-acaaaa0f6b2c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1347c0bd4ada4139ad194d423958be6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:02.040874 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57440e-a1cf-4fed-953d-acaaaa0f6b2c service glance] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:02.041268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57440e-a1cf-4fed-953d-acaaaa0f6b2c service glance] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:02.041466 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57440e-a1cf-4fed-953d-acaaaa0f6b2c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:02.049962 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db57440e-a1cf-4fed-953d-acaaaa0f6b2c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:02.051405 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db57440e-a1cf-4fed-953d-acaaaa0f6b2c service glance] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:02.060392 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 970/1917] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:02.204282 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:02.249982 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:02.621390 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:02.851041 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:03.065473 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbce6a7-aba3-4181-a2c1-c7dd91b3cd8f None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:03.066876 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-ecbce6a7-aba3-4181-a2c1-c7dd91b3cd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:03.139684 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db57440e-a1cf-4fed-953d-acaaaa0f6b2c service glance] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:03.141361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-db57440e-a1cf-4fed-953d-acaaaa0f6b2c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:03.145217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbce6a7-aba3-4181-a2c1-c7dd91b3cd8f None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:03.150190 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a667300-73c0-48c5-ab0b-67daefc137c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:03.151082 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a667300-73c0-48c5-ab0b-67daefc137c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:03.151394 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a667300-73c0-48c5-ab0b-67daefc137c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:03.151764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a667300-73c0-48c5-ab0b-67daefc137c7 admin admin] PATH_INFO: `/v3/users/9845796304594bad9d36935e8e24448e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:03.153030 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a667300-73c0-48c5-ab0b-67daefc137c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9845796304594bad9d36935e8e24448e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:03.155055 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a667300-73c0-48c5-ab0b-67daefc137c7 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:03.159408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:03.202609 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db57440e-a1cf-4fed-953d-acaaaa0f6b2c service glance] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:03.206318 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade0a21-e058-481d-87d2-c0f21469bc18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:03.207108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade0a21-e058-481d-87d2-c0f21469bc18 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:03.207466 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade0a21-e058-481d-87d2-c0f21469bc18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:03.209208 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade0a21-e058-481d-87d2-c0f21469bc18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:03.211326 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-aade0a21-e058-481d-87d2-c0f21469bc18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:03.279291 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade0a21-e058-481d-87d2-c0f21469bc18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:03.280823 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade0a21-e058-481d-87d2-c0f21469bc18 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:03.291217 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 971/1918] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:03 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:51:03.349175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-8a667300-73c0-48c5-ab0b-67daefc137c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9845796304594bad9d36935e8e24448e'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:03.363407 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-8a667300-73c0-48c5-ab0b-67daefc137c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9845796304594bad9d36935e8e24448e'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:03.444771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-8a667300-73c0-48c5-ab0b-67daefc137c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9845796304594bad9d36935e8e24448e'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:03.445827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-8a667300-73c0-48c5-ab0b-67daefc137c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9845796304594bad9d36935e8e24448e'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:03.445966 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:03.446535 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-8a667300-73c0-48c5-ab0b-67daefc137c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9845796304594bad9d36935e8e24448e'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:03.469468 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 948/1919] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:03 2022] DELETE /identity/v3/users/9845796304594bad9d36935e8e24448e => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:03.471056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ab3b2-2dcd-4cea-8658-8b5008c4a4a8 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:03.471356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ab3b2-2dcd-4cea-8658-8b5008c4a4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:03.471717 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ab3b2-2dcd-4cea-8658-8b5008c4a4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:03.492492 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-063ab3b2-2dcd-4cea-8658-8b5008c4a4a8 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:03.499548 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-063ab3b2-2dcd-4cea-8658-8b5008c4a4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:03.581136 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 972/1920] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:03.595603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a667300-73c0-48c5-ab0b-67daefc137c7 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:03.597192 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-8a667300-73c0-48c5-ab0b-67daefc137c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:03.660228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a667300-73c0-48c5-ab0b-67daefc137c7 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:03.663813 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e22fcd-8990-424e-bf7e-b8bc25f84035 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:03.664458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e22fcd-8990-424e-bf7e-b8bc25f84035 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:03.664742 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e22fcd-8990-424e-bf7e-b8bc25f84035 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:03.665790 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e22fcd-8990-424e-bf7e-b8bc25f84035 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:03.674967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e22fcd-8990-424e-bf7e-b8bc25f84035 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:03.676432 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e22fcd-8990-424e-bf7e-b8bc25f84035 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:03.686374 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 949/1921] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:04.228696 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:04.308832 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:04.316854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ab3b2-2dcd-4cea-8658-8b5008c4a4a8 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:04.318661 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-063ab3b2-2dcd-4cea-8658-8b5008c4a4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:04.340172 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ab3b2-2dcd-4cea-8658-8b5008c4a4a8 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:04.347236 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2f1d0-9dee-4e0b-8534-c7c1b829f690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:04.348153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2f1d0-9dee-4e0b-8534-c7c1b829f690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:04.348603 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2f1d0-9dee-4e0b-8534-c7c1b829f690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:04.349119 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2f1d0-9dee-4e0b-8534-c7c1b829f690 admin admin] PATH_INFO: `/v3/projects/f63fddb581024c34812bed88a9ad69d7` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:04.350390 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d2f1d0-9dee-4e0b-8534-c7c1b829f690 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f63fddb581024c34812bed88a9ad69d7)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:04.353909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d2f1d0-9dee-4e0b-8534-c7c1b829f690 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:04.533245 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-91d2f1d0-9dee-4e0b-8534-c7c1b829f690 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f63fddb581024c34812bed88a9ad69d7', 'request_id': 'req-91d2f1d0-9dee-4e0b-8534-c7c1b829f690'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:04.539232 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-91d2f1d0-9dee-4e0b-8534-c7c1b829f690 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f63fddb581024c34812bed88a9ad69d7', 'request_id': 'req-91d2f1d0-9dee-4e0b-8534-c7c1b829f690'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:04.546963 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-91d2f1d0-9dee-4e0b-8534-c7c1b829f690 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f63fddb581024c34812bed88a9ad69d7', 'request_id': 'req-91d2f1d0-9dee-4e0b-8534-c7c1b829f690'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:04.615120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-91d2f1d0-9dee-4e0b-8534-c7c1b829f690 admin admin] The token cache is being invalidate because project f63fddb581024c34812bed88a9ad69d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:51:04.615417 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-91d2f1d0-9dee-4e0b-8534-c7c1b829f690 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:04.616231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-91d2f1d0-9dee-4e0b-8534-c7c1b829f690 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:04.618119 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 973/1922] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:04 2022] DELETE /identity/v3/projects/f63fddb581024c34812bed88a9ad69d7 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:51:04.622219 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e22fcd-8990-424e-bf7e-b8bc25f84035 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:04.632937 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e22fcd-8990-424e-bf7e-b8bc25f84035 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:04.715102 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17593085-7e1d-473f-8185-43392f4f543b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:04.716364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17593085-7e1d-473f-8185-43392f4f543b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:04.716684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17593085-7e1d-473f-8185-43392f4f543b admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:04.717112 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17593085-7e1d-473f-8185-43392f4f543b admin admin] PATH_INFO: `/v3/users/3cff73dd9c084ea29792bb83f012edeb` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:04.723966 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17593085-7e1d-473f-8185-43392f4f543b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cff73dd9c084ea29792bb83f012edeb)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:04.723966 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17593085-7e1d-473f-8185-43392f4f543b admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:04.944986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-17593085-7e1d-473f-8185-43392f4f543b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cff73dd9c084ea29792bb83f012edeb'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:04.949104 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:04.961817 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-17593085-7e1d-473f-8185-43392f4f543b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cff73dd9c084ea29792bb83f012edeb'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:05.066026 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-17593085-7e1d-473f-8185-43392f4f543b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cff73dd9c084ea29792bb83f012edeb'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:05.066657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-17593085-7e1d-473f-8185-43392f4f543b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cff73dd9c084ea29792bb83f012edeb'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:05.067360 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-17593085-7e1d-473f-8185-43392f4f543b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cff73dd9c084ea29792bb83f012edeb'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:05.082222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 950/1923] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:04 2022] DELETE /identity/v3/users/3cff73dd9c084ea29792bb83f012edeb => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:05.561821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2f1d0-9dee-4e0b-8534-c7c1b829f690 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:05.565185 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-91d2f1d0-9dee-4e0b-8534-c7c1b829f690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:05.581248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2f1d0-9dee-4e0b-8534-c7c1b829f690 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:05.625858 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:05.664470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff65d4b-490d-45f6-add0-fa61142f8155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:05.666009 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff65d4b-490d-45f6-add0-fa61142f8155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:05.666306 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff65d4b-490d-45f6-add0-fa61142f8155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:05.666591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff65d4b-490d-45f6-add0-fa61142f8155 admin admin] PATH_INFO: `/v3/projects/2ad70bc1d6d64d1ea0a864d5b3ef231c` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:05.667846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff65d4b-490d-45f6-add0-fa61142f8155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ad70bc1d6d64d1ea0a864d5b3ef231c)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:05.669843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff65d4b-490d-45f6-add0-fa61142f8155 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:05.784672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-0ff65d4b-490d-45f6-add0-fa61142f8155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ad70bc1d6d64d1ea0a864d5b3ef231c', 'request_id': 'req-0ff65d4b-490d-45f6-add0-fa61142f8155'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:05.790399 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-0ff65d4b-490d-45f6-add0-fa61142f8155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ad70bc1d6d64d1ea0a864d5b3ef231c', 'request_id': 'req-0ff65d4b-490d-45f6-add0-fa61142f8155'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:05.797983 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-0ff65d4b-490d-45f6-add0-fa61142f8155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ad70bc1d6d64d1ea0a864d5b3ef231c', 'request_id': 'req-0ff65d4b-490d-45f6-add0-fa61142f8155'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:05.906743 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-0ff65d4b-490d-45f6-add0-fa61142f8155 admin admin] The token cache is being invalidate because project 2ad70bc1d6d64d1ea0a864d5b3ef231c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:51:05.907032 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-0ff65d4b-490d-45f6-add0-fa61142f8155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:05.909010 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-0ff65d4b-490d-45f6-add0-fa61142f8155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:05.917198 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 974/1924] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:05 2022] DELETE /identity/v3/projects/2ad70bc1d6d64d1ea0a864d5b3ef231c => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:05.922248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7073d-82ee-41b7-b77c-6c56fe070066 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:05.922701 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7073d-82ee-41b7-b77c-6c56fe070066 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:05.923017 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb7073d-82ee-41b7-b77c-6c56fe070066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:05.955915 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-4eb7073d-82ee-41b7-b77c-6c56fe070066 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:05.957343 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-4eb7073d-82ee-41b7-b77c-6c56fe070066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:06.028994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 951/1925] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:06.035346 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b6d47-5613-411d-86fc-10cd5d0bff21 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:06.035642 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b6d47-5613-411d-86fc-10cd5d0bff21 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:06.035921 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b6d47-5613-411d-86fc-10cd5d0bff21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:06.066477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-063b6d47-5613-411d-86fc-10cd5d0bff21 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:06.066477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-063b6d47-5613-411d-86fc-10cd5d0bff21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:06.073405 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 975/1926] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:06.079286 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7073d-82ee-41b7-b77c-6c56fe070066 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:06.089928 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb7073d-82ee-41b7-b77c-6c56fe070066 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:06.095169 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc496cb-eb85-48dc-b18b-16ff8cbab956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:06.095852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc496cb-eb85-48dc-b18b-16ff8cbab956 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:06.096142 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc496cb-eb85-48dc-b18b-16ff8cbab956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:06.096425 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc496cb-eb85-48dc-b18b-16ff8cbab956 admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:06.097201 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc496cb-eb85-48dc-b18b-16ff8cbab956 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:51:06.097516 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc496cb-eb85-48dc-b18b-16ff8cbab956 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:06.099289 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc496cb-eb85-48dc-b18b-16ff8cbab956 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:06.116241 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 952/1927] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:51:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:51:06.117686 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b6d47-5613-411d-86fc-10cd5d0bff21 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:06.128839 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b6d47-5613-411d-86fc-10cd5d0bff21 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:06.138001 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ae6c4-698f-4ca5-8993-79b1d3e2a531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:06.138703 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ae6c4-698f-4ca5-8993-79b1d3e2a531 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:06.138977 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ae6c4-698f-4ca5-8993-79b1d3e2a531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:06.139255 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2ae6c4-698f-4ca5-8993-79b1d3e2a531 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:06.141300 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2ae6c4-698f-4ca5-8993-79b1d3e2a531 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:06.147406 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2ae6c4-698f-4ca5-8993-79b1d3e2a531 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:06.254429 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 976/1928] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:51:06 2022] POST /identity/v3/projects => generated 389 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:06.256274 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc496cb-eb85-48dc-b18b-16ff8cbab956 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:06.266498 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc496cb-eb85-48dc-b18b-16ff8cbab956 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:06.332077 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5fb59f-dccf-4a54-9b8b-10b45f572c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:06.332939 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5fb59f-dccf-4a54-9b8b-10b45f572c92 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:06.333472 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5fb59f-dccf-4a54-9b8b-10b45f572c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:06.334053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5fb59f-dccf-4a54-9b8b-10b45f572c92 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:06.338192 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5fb59f-dccf-4a54-9b8b-10b45f572c92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:06.338192 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5fb59f-dccf-4a54-9b8b-10b45f572c92 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:06.480406 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 953/1929] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:51:06 2022] POST /identity/v3/users => generated 295 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:51:06.484367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ae6c4-698f-4ca5-8993-79b1d3e2a531 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:06.494733 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2ae6c4-698f-4ca5-8993-79b1d3e2a531 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:06.500121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640ffb1-31a2-4e8d-8c85-8258d2a23077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:06.501088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640ffb1-31a2-4e8d-8c85-8258d2a23077 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:06.501393 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640ffb1-31a2-4e8d-8c85-8258d2a23077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:06.501675 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640ffb1-31a2-4e8d-8c85-8258d2a23077 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:06.502448 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640ffb1-31a2-4e8d-8c85-8258d2a23077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:06.504115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640ffb1-31a2-4e8d-8c85-8258d2a23077 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:06.516745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 977/1930] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:06.519470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5fb59f-dccf-4a54-9b8b-10b45f572c92 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:06.531020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5fb59f-dccf-4a54-9b8b-10b45f572c92 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:06.537513 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86c0bb-26ae-4b67-83b5-19895552be4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:06.538558 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86c0bb-26ae-4b67-83b5-19895552be4d admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:06.539033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86c0bb-26ae-4b67-83b5-19895552be4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:06.539482 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86c0bb-26ae-4b67-83b5-19895552be4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:06.540903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86c0bb-26ae-4b67-83b5-19895552be4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:06.542783 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86c0bb-26ae-4b67-83b5-19895552be4d admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:06.568143 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 954/1931] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:06 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:06.572453 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640ffb1-31a2-4e8d-8c85-8258d2a23077 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:06.575125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:06.583816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640ffb1-31a2-4e8d-8c85-8258d2a23077 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:06.589690 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2d4fa-37fb-4cf0-a66c-93ae13497128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:06.590743 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2d4fa-37fb-4cf0-a66c-93ae13497128 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:06.591026 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2d4fa-37fb-4cf0-a66c-93ae13497128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:06.591325 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb2d4fa-37fb-4cf0-a66c-93ae13497128 admin admin] PATH_INFO: `/v3/projects/1fd8a65ccbac4c42920114d81296ed33/users/912e84ca8bce4502bef7bf277edeba4b/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:06.611026 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb2d4fa-37fb-4cf0-a66c-93ae13497128 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fd8a65ccbac4c42920114d81296ed33, user_id=912e84ca8bce4502bef7bf277edeba4b, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:06.614466 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb2d4fa-37fb-4cf0-a66c-93ae13497128 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:06.679320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 978/1932] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:06 2022] PUT /identity/v3/projects/1fd8a65ccbac4c42920114d81296ed33/users/912e84ca8bce4502bef7bf277edeba4b/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:06.689989 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f54f74-65a8-49d8-b5a5-fff09af376de None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:06.690330 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f54f74-65a8-49d8-b5a5-fff09af376de None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:06.690618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f54f74-65a8-49d8-b5a5-fff09af376de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:06.724375 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-96f54f74-65a8-49d8-b5a5-fff09af376de None None] MFA Rules not processed for user `912e84ca8bce4502bef7bf277edeba4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:06.726348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-96f54f74-65a8-49d8-b5a5-fff09af376de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:06.796263 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 955/1933] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:06.801113 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2d4fa-37fb-4cf0-a66c-93ae13497128 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:06.865631 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb2d4fa-37fb-4cf0-a66c-93ae13497128 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:06.870905 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5db6d-d863-4382-81c3-b9f8cfaa35d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:06.871632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5db6d-d863-4382-81c3-b9f8cfaa35d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:06.871919 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5db6d-d863-4382-81c3-b9f8cfaa35d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:06.872194 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a5db6d-d863-4382-81c3-b9f8cfaa35d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:06.872880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a5db6d-d863-4382-81c3-b9f8cfaa35d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:06.875169 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a5db6d-d863-4382-81c3-b9f8cfaa35d3 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:06.967704 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 979/1934] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:51:06 2022] POST /identity/v3/projects => generated 393 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:06.972294 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f54f74-65a8-49d8-b5a5-fff09af376de None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:06.987368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f54f74-65a8-49d8-b5a5-fff09af376de None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:07.051013 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00ac72-0ad0-41cb-8658-9814904d5ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:07.051726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00ac72-0ad0-41cb-8658-9814904d5ca3 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:07.052021 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00ac72-0ad0-41cb-8658-9814904d5ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:07.052317 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00ac72-0ad0-41cb-8658-9814904d5ca3 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:07.053203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00ac72-0ad0-41cb-8658-9814904d5ca3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:07.055141 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00ac72-0ad0-41cb-8658-9814904d5ca3 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:07.137140 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 956/1935] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:51:06 2022] POST /identity/v3/users => generated 303 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:51:07.141305 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5db6d-d863-4382-81c3-b9f8cfaa35d3 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:07.153407 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a5db6d-d863-4382-81c3-b9f8cfaa35d3 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:07.158377 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a438db0-21a6-47ab-afc6-88efef3184ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:07.159068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a438db0-21a6-47ab-afc6-88efef3184ec admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:07.159370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a438db0-21a6-47ab-afc6-88efef3184ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:07.159648 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a438db0-21a6-47ab-afc6-88efef3184ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:07.160434 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a438db0-21a6-47ab-afc6-88efef3184ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:07.162065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a438db0-21a6-47ab-afc6-88efef3184ec admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:07.174230 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 980/1936] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:07.178478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00ac72-0ad0-41cb-8658-9814904d5ca3 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:07.190846 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00ac72-0ad0-41cb-8658-9814904d5ca3 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:07.195720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fef1fe-3cd5-434e-8ecc-2a1f9570ca53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:07.196945 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fef1fe-3cd5-434e-8ecc-2a1f9570ca53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:07.197252 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fef1fe-3cd5-434e-8ecc-2a1f9570ca53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:07.197532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fef1fe-3cd5-434e-8ecc-2a1f9570ca53 admin admin] PATH_INFO: `/v3/projects/87463c7045214776845739dbdafbe695/users/8f2b86927c39495681e54ffafbf4823e/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:07.213555 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fef1fe-3cd5-434e-8ecc-2a1f9570ca53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87463c7045214776845739dbdafbe695, user_id=8f2b86927c39495681e54ffafbf4823e, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:07.216284 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fef1fe-3cd5-434e-8ecc-2a1f9570ca53 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:07.311812 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 957/1937] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:07 2022] PUT /identity/v3/projects/87463c7045214776845739dbdafbe695/users/8f2b86927c39495681e54ffafbf4823e/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:07.315923 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a438db0-21a6-47ab-afc6-88efef3184ec admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:07.329284 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a438db0-21a6-47ab-afc6-88efef3184ec admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:07.436649 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4968e-48b5-4cb1-b0a0-de7bf67d2228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:07.437374 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4968e-48b5-4cb1-b0a0-de7bf67d2228 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:07.437659 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4968e-48b5-4cb1-b0a0-de7bf67d2228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:07.438010 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4968e-48b5-4cb1-b0a0-de7bf67d2228 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:07.446677 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb4968e-48b5-4cb1-b0a0-de7bf67d2228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:07.449157 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb4968e-48b5-4cb1-b0a0-de7bf67d2228 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:07.470493 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 981/1938] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:07 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:07.474401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fef1fe-3cd5-434e-8ecc-2a1f9570ca53 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:07.492123 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fef1fe-3cd5-434e-8ecc-2a1f9570ca53 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:07.502039 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4afbb-dd19-408d-9c57-fe6cb0d0202b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:07.502039 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4afbb-dd19-408d-9c57-fe6cb0d0202b admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:07.502338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4afbb-dd19-408d-9c57-fe6cb0d0202b admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:07.502567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4afbb-dd19-408d-9c57-fe6cb0d0202b admin admin] PATH_INFO: `/v3/projects/87463c7045214776845739dbdafbe695/users/8f2b86927c39495681e54ffafbf4823e/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:07.504891 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4afbb-dd19-408d-9c57-fe6cb0d0202b admin admin] RBAC: Authorizing `identity:create_grant(project_id=87463c7045214776845739dbdafbe695, user_id=8f2b86927c39495681e54ffafbf4823e, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:07.509024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4afbb-dd19-408d-9c57-fe6cb0d0202b admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:07.540423 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 958/1939] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:07 2022] PUT /identity/v3/projects/87463c7045214776845739dbdafbe695/users/8f2b86927c39495681e54ffafbf4823e/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:07.562765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5f922-ec81-41d0-afc9-ed5403290e33 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:07.563292 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5f922-ec81-41d0-afc9-ed5403290e33 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:07.563687 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb5f922-ec81-41d0-afc9-ed5403290e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:07.625368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-6cb5f922-ec81-41d0-afc9-ed5403290e33 None None] MFA Rules not processed for user `8f2b86927c39495681e54ffafbf4823e`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:07.625368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-6cb5f922-ec81-41d0-afc9-ed5403290e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:07.700086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:07.704102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 982/1940] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:07.715563 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4afbb-dd19-408d-9c57-fe6cb0d0202b admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:07.717355 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-b1c4afbb-dd19-408d-9c57-fe6cb0d0202b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:07.777799 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4afbb-dd19-408d-9c57-fe6cb0d0202b admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:07.789327 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71126304-ab35-4724-bf40-375f23741ebb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:07.790037 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71126304-ab35-4724-bf40-375f23741ebb service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:07.790376 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71126304-ab35-4724-bf40-375f23741ebb service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:07.790730 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71126304-ab35-4724-bf40-375f23741ebb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:07.801193 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71126304-ab35-4724-bf40-375f23741ebb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:07.802630 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71126304-ab35-4724-bf40-375f23741ebb service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:07.812841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 959/1941] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:07 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:07.870025 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb5f922-ec81-41d0-afc9-ed5403290e33 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:07.871472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-6cb5f922-ec81-41d0-afc9-ed5403290e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:07.884439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb5f922-ec81-41d0-afc9-ed5403290e33 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:07.888205 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c1687d-932c-4a4e-9f83-bde92536e4c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:07.888981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1687d-932c-4a4e-9f83-bde92536e4c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:07.889320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1687d-932c-4a4e-9f83-bde92536e4c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:07.889623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c1687d-932c-4a4e-9f83-bde92536e4c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:07.957825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c1687d-932c-4a4e-9f83-bde92536e4c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:07.957825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c1687d-932c-4a4e-9f83-bde92536e4c2 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:07.965489 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 983/1942] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:51:08.718042 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:09.312269 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:14.284125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71126304-ab35-4724-bf40-375f23741ebb service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:14.354917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71126304-ab35-4724-bf40-375f23741ebb service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:14.364398 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c119c6-b236-4bc2-abfb-9c1e84f62a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:14.364398 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c119c6-b236-4bc2-abfb-9c1e84f62a4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:14.364398 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c119c6-b236-4bc2-abfb-9c1e84f62a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:14.364398 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c119c6-b236-4bc2-abfb-9c1e84f62a4a admin admin] PATH_INFO: `/v3/users/912e84ca8bce4502bef7bf277edeba4b` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:14.364756 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c119c6-b236-4bc2-abfb-9c1e84f62a4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=912e84ca8bce4502bef7bf277edeba4b)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:14.364756 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c119c6-b236-4bc2-abfb-9c1e84f62a4a admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:14.574182 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:14.579598 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f3c119c6-b236-4bc2-abfb-9c1e84f62a4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '912e84ca8bce4502bef7bf277edeba4b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:14.594249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f3c119c6-b236-4bc2-abfb-9c1e84f62a4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '912e84ca8bce4502bef7bf277edeba4b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:14.676822 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f3c119c6-b236-4bc2-abfb-9c1e84f62a4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '912e84ca8bce4502bef7bf277edeba4b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:14.676822 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f3c119c6-b236-4bc2-abfb-9c1e84f62a4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '912e84ca8bce4502bef7bf277edeba4b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:14.676822 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f3c119c6-b236-4bc2-abfb-9c1e84f62a4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '912e84ca8bce4502bef7bf277edeba4b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:14.687178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 960/1943] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:14 2022] DELETE /identity/v3/users/912e84ca8bce4502bef7bf277edeba4b => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:14.691776 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac1656-0f20-4a4f-9fa3-618cc2427ece None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:14.692295 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac1656-0f20-4a4f-9fa3-618cc2427ece None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:14.692396 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac1656-0f20-4a4f-9fa3-618cc2427ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:14.710931 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-16ac1656-0f20-4a4f-9fa3-618cc2427ece None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:14.712754 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-16ac1656-0f20-4a4f-9fa3-618cc2427ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:14.777686 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 984/1944] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:14.789007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c119c6-b236-4bc2-abfb-9c1e84f62a4a admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:14.797881 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-f3c119c6-b236-4bc2-abfb-9c1e84f62a4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:14.858796 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c119c6-b236-4bc2-abfb-9c1e84f62a4a admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:14.862725 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d1f21-fe24-4731-b577-0a3fc18fb0a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:14.863387 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d1f21-fe24-4731-b577-0a3fc18fb0a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:14.863660 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d1f21-fe24-4731-b577-0a3fc18fb0a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:14.863942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d1f21-fe24-4731-b577-0a3fc18fb0a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:14.873931 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415d1f21-fe24-4731-b577-0a3fc18fb0a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:14.875386 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415d1f21-fe24-4731-b577-0a3fc18fb0a1 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:14.885495 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 961/1945] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:51:15.326027 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac1656-0f20-4a4f-9fa3-618cc2427ece None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:15.327468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-16ac1656-0f20-4a4f-9fa3-618cc2427ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:15.340630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac1656-0f20-4a4f-9fa3-618cc2427ece None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:15.348037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f87db-03af-43b3-83b2-182431611e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:15.350670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f87db-03af-43b3-83b2-182431611e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:15.350670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f87db-03af-43b3-83b2-182431611e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:15.350670 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f87db-03af-43b3-83b2-182431611e0d admin admin] PATH_INFO: `/v3/projects/1fd8a65ccbac4c42920114d81296ed33` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:15.352631 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f87db-03af-43b3-83b2-182431611e0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fd8a65ccbac4c42920114d81296ed33)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:15.356623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f87db-03af-43b3-83b2-182431611e0d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:15.495629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d10f87db-03af-43b3-83b2-182431611e0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fd8a65ccbac4c42920114d81296ed33', 'request_id': 'req-d10f87db-03af-43b3-83b2-182431611e0d'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:15.501093 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d10f87db-03af-43b3-83b2-182431611e0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fd8a65ccbac4c42920114d81296ed33', 'request_id': 'req-d10f87db-03af-43b3-83b2-182431611e0d'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:15.507351 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d10f87db-03af-43b3-83b2-182431611e0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fd8a65ccbac4c42920114d81296ed33', 'request_id': 'req-d10f87db-03af-43b3-83b2-182431611e0d'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:15.575224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d10f87db-03af-43b3-83b2-182431611e0d admin admin] The token cache is being invalidate because project 1fd8a65ccbac4c42920114d81296ed33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:51:15.575510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d10f87db-03af-43b3-83b2-182431611e0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:15.576139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d10f87db-03af-43b3-83b2-182431611e0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:15.577731 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 985/1946] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:15 2022] DELETE /identity/v3/projects/1fd8a65ccbac4c42920114d81296ed33 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:15.581856 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d1f21-fe24-4731-b577-0a3fc18fb0a1 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:15.592099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d1f21-fe24-4731-b577-0a3fc18fb0a1 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:15.652676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439238e-4f63-4710-994d-67e1a11880b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:15.653597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439238e-4f63-4710-994d-67e1a11880b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:15.653883 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439238e-4f63-4710-994d-67e1a11880b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:15.654162 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439238e-4f63-4710-994d-67e1a11880b3 admin admin] PATH_INFO: `/v3/users/8f2b86927c39495681e54ffafbf4823e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:15.655350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439238e-4f63-4710-994d-67e1a11880b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f2b86927c39495681e54ffafbf4823e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:15.657297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439238e-4f63-4710-994d-67e1a11880b3 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:15.829512 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-e439238e-4f63-4710-994d-67e1a11880b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f2b86927c39495681e54ffafbf4823e'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:15.841044 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:15.849011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-e439238e-4f63-4710-994d-67e1a11880b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f2b86927c39495681e54ffafbf4823e'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:15.906561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-e439238e-4f63-4710-994d-67e1a11880b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f2b86927c39495681e54ffafbf4823e'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:15.907186 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-e439238e-4f63-4710-994d-67e1a11880b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f2b86927c39495681e54ffafbf4823e'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:15.907800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-e439238e-4f63-4710-994d-67e1a11880b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f2b86927c39495681e54ffafbf4823e'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:15.920290 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 962/1947] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:15 2022] DELETE /identity/v3/users/8f2b86927c39495681e54ffafbf4823e => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:16.363973 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f87db-03af-43b3-83b2-182431611e0d admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:16.373713 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d10f87db-03af-43b3-83b2-182431611e0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:16.379063 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f87db-03af-43b3-83b2-182431611e0d admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:16.438842 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5a6b2c-1295-40a6-96ef-09f5c8f9c450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:16.439724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a6b2c-1295-40a6-96ef-09f5c8f9c450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:16.440011 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a6b2c-1295-40a6-96ef-09f5c8f9c450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:16.440324 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a6b2c-1295-40a6-96ef-09f5c8f9c450 admin admin] PATH_INFO: `/v3/projects/87463c7045214776845739dbdafbe695` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:16.441501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a6b2c-1295-40a6-96ef-09f5c8f9c450 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87463c7045214776845739dbdafbe695)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:16.443739 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5a6b2c-1295-40a6-96ef-09f5c8f9c450 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:16.471700 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:16.559219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-3c5a6b2c-1295-40a6-96ef-09f5c8f9c450 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87463c7045214776845739dbdafbe695', 'request_id': 'req-3c5a6b2c-1295-40a6-96ef-09f5c8f9c450'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:16.564554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-3c5a6b2c-1295-40a6-96ef-09f5c8f9c450 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87463c7045214776845739dbdafbe695', 'request_id': 'req-3c5a6b2c-1295-40a6-96ef-09f5c8f9c450'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:16.571531 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-3c5a6b2c-1295-40a6-96ef-09f5c8f9c450 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87463c7045214776845739dbdafbe695', 'request_id': 'req-3c5a6b2c-1295-40a6-96ef-09f5c8f9c450'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:16.638578 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-3c5a6b2c-1295-40a6-96ef-09f5c8f9c450 admin admin] The token cache is being invalidate because project 87463c7045214776845739dbdafbe695 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:51:16.638867 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-3c5a6b2c-1295-40a6-96ef-09f5c8f9c450 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:16.639495 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-3c5a6b2c-1295-40a6-96ef-09f5c8f9c450 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:16.641065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 986/1948] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:16 2022] DELETE /identity/v3/projects/87463c7045214776845739dbdafbe695 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:16.649259 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55568ac4-9149-4084-8e16-e80f55d3b2f0 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:16.649259 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55568ac4-9149-4084-8e16-e80f55d3b2f0 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:16.649259 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55568ac4-9149-4084-8e16-e80f55d3b2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:16.671950 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-55568ac4-9149-4084-8e16-e80f55d3b2f0 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:16.673176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-55568ac4-9149-4084-8e16-e80f55d3b2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:16.730994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 963/1949] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:16.738738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696616c9-b8a5-4905-bc36-e354be9640ef None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:16.739013 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696616c9-b8a5-4905-bc36-e354be9640ef None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:16.739296 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696616c9-b8a5-4905-bc36-e354be9640ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:16.760223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-696616c9-b8a5-4905-bc36-e354be9640ef None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:16.761464 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-696616c9-b8a5-4905-bc36-e354be9640ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:16.769415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 987/1950] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:16.776847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55568ac4-9149-4084-8e16-e80f55d3b2f0 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:16.786455 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55568ac4-9149-4084-8e16-e80f55d3b2f0 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:16.791944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b1bff-02ba-48af-b6ce-23b0f1d66e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:16.792614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b1bff-02ba-48af-b6ce-23b0f1d66e5a admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:16.792899 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b1bff-02ba-48af-b6ce-23b0f1d66e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:16.793212 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b1bff-02ba-48af-b6ce-23b0f1d66e5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:16.793953 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1b1bff-02ba-48af-b6ce-23b0f1d66e5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:51:16.794256 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1b1bff-02ba-48af-b6ce-23b0f1d66e5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:16.795845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1b1bff-02ba-48af-b6ce-23b0f1d66e5a admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:16.811901 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 964/1951] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:51:16.813325 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696616c9-b8a5-4905-bc36-e354be9640ef None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:16.823214 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696616c9-b8a5-4905-bc36-e354be9640ef None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:16.828018 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439a316-9940-4aa4-89fd-b24389f815b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:16.828668 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439a316-9940-4aa4-89fd-b24389f815b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:16.828969 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439a316-9940-4aa4-89fd-b24389f815b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:16.829284 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439a316-9940-4aa4-89fd-b24389f815b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:16.829967 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8439a316-9940-4aa4-89fd-b24389f815b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:16.831897 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8439a316-9940-4aa4-89fd-b24389f815b4 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:16.920070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 988/1952] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:51:16 2022] POST /identity/v3/projects => generated 407 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:16.924258 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b1bff-02ba-48af-b6ce-23b0f1d66e5a admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:16.934842 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b1bff-02ba-48af-b6ce-23b0f1d66e5a admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:16.990144 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061fdc5c-f4c9-468c-94d5-17e1c637aec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:16.990833 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fdc5c-f4c9-468c-94d5-17e1c637aec9 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:16.991101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fdc5c-f4c9-468c-94d5-17e1c637aec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:16.991379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fdc5c-f4c9-468c-94d5-17e1c637aec9 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:16.992063 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061fdc5c-f4c9-468c-94d5-17e1c637aec9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:16.994273 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061fdc5c-f4c9-468c-94d5-17e1c637aec9 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:17.096089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 965/1953] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:51:16 2022] POST /identity/v3/users => generated 304 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:17.100038 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439a316-9940-4aa4-89fd-b24389f815b4 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:17.111864 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439a316-9940-4aa4-89fd-b24389f815b4 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:17.116714 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e49c0c2-71f8-4e0a-942d-4a8924b84ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:17.117452 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49c0c2-71f8-4e0a-942d-4a8924b84ae7 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:17.117909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49c0c2-71f8-4e0a-942d-4a8924b84ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:17.118289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49c0c2-71f8-4e0a-942d-4a8924b84ae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:17.119041 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e49c0c2-71f8-4e0a-942d-4a8924b84ae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:17.120659 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e49c0c2-71f8-4e0a-942d-4a8924b84ae7 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:17.131627 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 989/1954] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:17.135834 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061fdc5c-f4c9-468c-94d5-17e1c637aec9 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:17.146170 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061fdc5c-f4c9-468c-94d5-17e1c637aec9 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:17.151544 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898508f-c2e8-406e-a4bb-4026b7d55c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:17.152205 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898508f-c2e8-406e-a4bb-4026b7d55c20 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:17.155741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898508f-c2e8-406e-a4bb-4026b7d55c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:17.155741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898508f-c2e8-406e-a4bb-4026b7d55c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:17.155741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898508f-c2e8-406e-a4bb-4026b7d55c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:17.155741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898508f-c2e8-406e-a4bb-4026b7d55c20 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:17.170477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 966/1955] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:17.174511 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e49c0c2-71f8-4e0a-942d-4a8924b84ae7 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:17.184727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e49c0c2-71f8-4e0a-942d-4a8924b84ae7 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:17.189373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ede7e-c5d1-4dd1-86bc-6a3b7c9e072b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:17.190256 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ede7e-c5d1-4dd1-86bc-6a3b7c9e072b admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:17.190538 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ede7e-c5d1-4dd1-86bc-6a3b7c9e072b admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:17.190815 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ede7e-c5d1-4dd1-86bc-6a3b7c9e072b admin admin] PATH_INFO: `/v3/projects/e614399115ec4911a8557fde2d4b55f2/users/2801017cd40a45c2b9c82eec0c759717/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:17.205052 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ede7e-c5d1-4dd1-86bc-6a3b7c9e072b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e614399115ec4911a8557fde2d4b55f2, user_id=2801017cd40a45c2b9c82eec0c759717, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:17.207723 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ede7e-c5d1-4dd1-86bc-6a3b7c9e072b admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:17.278889 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 990/1956] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:17 2022] PUT /identity/v3/projects/e614399115ec4911a8557fde2d4b55f2/users/2801017cd40a45c2b9c82eec0c759717/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:17.290880 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f51750-4659-4c92-bbcc-35ede0906867 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:17.291222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f51750-4659-4c92-bbcc-35ede0906867 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:17.291533 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f51750-4659-4c92-bbcc-35ede0906867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:17.318864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-80f51750-4659-4c92-bbcc-35ede0906867 None None] MFA Rules not processed for user `2801017cd40a45c2b9c82eec0c759717`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:17.320105 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-80f51750-4659-4c92-bbcc-35ede0906867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:17.354618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:17.386494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 967/1957] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:17 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:17.391055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ede7e-c5d1-4dd1-86bc-6a3b7c9e072b admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:17.456013 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ede7e-c5d1-4dd1-86bc-6a3b7c9e072b admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:17.460751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436b627-3d3b-4b27-bf2c-5329cf37c7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:17.461581 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436b627-3d3b-4b27-bf2c-5329cf37c7a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:17.462085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436b627-3d3b-4b27-bf2c-5329cf37c7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:17.462374 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436b627-3d3b-4b27-bf2c-5329cf37c7a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:17.463088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c436b627-3d3b-4b27-bf2c-5329cf37c7a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:17.465044 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c436b627-3d3b-4b27-bf2c-5329cf37c7a8 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:17.596581 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 991/1958] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:51:17 2022] POST /identity/v3/projects => generated 409 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:51:17.600342 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f51750-4659-4c92-bbcc-35ede0906867 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:17.614266 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f51750-4659-4c92-bbcc-35ede0906867 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:17.677030 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484cc5f-98f3-49f1-8299-5c828b866fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:17.677724 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484cc5f-98f3-49f1-8299-5c828b866fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:17.677998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484cc5f-98f3-49f1-8299-5c828b866fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:17.678273 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0484cc5f-98f3-49f1-8299-5c828b866fc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:17.678985 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0484cc5f-98f3-49f1-8299-5c828b866fc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:17.681627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0484cc5f-98f3-49f1-8299-5c828b866fc4 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:17.766427 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 968/1959] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:51:17 2022] POST /identity/v3/users => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:51:17.769683 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436b627-3d3b-4b27-bf2c-5329cf37c7a8 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:17.780610 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436b627-3d3b-4b27-bf2c-5329cf37c7a8 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:17.785733 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04369614-83aa-4ea5-aed5-54e4dad49d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:17.789435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04369614-83aa-4ea5-aed5-54e4dad49d2a admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:17.789435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04369614-83aa-4ea5-aed5-54e4dad49d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:17.789435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04369614-83aa-4ea5-aed5-54e4dad49d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:17.789435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04369614-83aa-4ea5-aed5-54e4dad49d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:17.789435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04369614-83aa-4ea5-aed5-54e4dad49d2a admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:17.800020 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 992/1960] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:17.804739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484cc5f-98f3-49f1-8299-5c828b866fc4 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:17.816319 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0484cc5f-98f3-49f1-8299-5c828b866fc4 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:17.822277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ad21c-e93f-4661-8e71-070f169e9399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:17.823213 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ad21c-e93f-4661-8e71-070f169e9399 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:17.823527 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ad21c-e93f-4661-8e71-070f169e9399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:17.823880 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ad21c-e93f-4661-8e71-070f169e9399 admin admin] PATH_INFO: `/v3/projects/76ba3673912842d3b9ab1fdaba193e9d/users/e74d0fc799e2402686e4360952a4930a/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:17.839432 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ad21c-e93f-4661-8e71-070f169e9399 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76ba3673912842d3b9ab1fdaba193e9d, user_id=e74d0fc799e2402686e4360952a4930a, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:17.842142 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ad21c-e93f-4661-8e71-070f169e9399 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:18.029754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 969/1961] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:17 2022] PUT /identity/v3/projects/76ba3673912842d3b9ab1fdaba193e9d/users/e74d0fc799e2402686e4360952a4930a/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:18.034019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04369614-83aa-4ea5-aed5-54e4dad49d2a admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:18.044873 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04369614-83aa-4ea5-aed5-54e4dad49d2a admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:18.070057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:18.112742 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50b7a7e-46d4-430b-b46d-fbe360eb749a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:18.113884 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b7a7e-46d4-430b-b46d-fbe360eb749a admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:18.114178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b7a7e-46d4-430b-b46d-fbe360eb749a admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:18.114458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50b7a7e-46d4-430b-b46d-fbe360eb749a admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:18.115191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b7a7e-46d4-430b-b46d-fbe360eb749a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:18.116799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50b7a7e-46d4-430b-b46d-fbe360eb749a admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:18.130898 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 993/1962] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:18 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:18.131921 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ad21c-e93f-4661-8e71-070f169e9399 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:18.147279 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ad21c-e93f-4661-8e71-070f169e9399 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:18.152202 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7fc17-7e23-44e5-af5b-7b6f4727b104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:18.158442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7fc17-7e23-44e5-af5b-7b6f4727b104 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:18.158442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7fc17-7e23-44e5-af5b-7b6f4727b104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:18.158442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac7fc17-7e23-44e5-af5b-7b6f4727b104 admin admin] PATH_INFO: `/v3/projects/76ba3673912842d3b9ab1fdaba193e9d/users/e74d0fc799e2402686e4360952a4930a/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:18.159272 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7fc17-7e23-44e5-af5b-7b6f4727b104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76ba3673912842d3b9ab1fdaba193e9d, user_id=e74d0fc799e2402686e4360952a4930a, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:18.162260 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac7fc17-7e23-44e5-af5b-7b6f4727b104 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:18.174982 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 970/1963] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:18 2022] PUT /identity/v3/projects/76ba3673912842d3b9ab1fdaba193e9d/users/e74d0fc799e2402686e4360952a4930a/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:18.190255 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196048a-2986-4eba-b47d-dd1f4b83e28a None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:18.190559 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196048a-2986-4eba-b47d-dd1f4b83e28a None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:18.190839 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196048a-2986-4eba-b47d-dd1f4b83e28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:18.218545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-d196048a-2986-4eba-b47d-dd1f4b83e28a None None] MFA Rules not processed for user `e74d0fc799e2402686e4360952a4930a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:18.219883 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d196048a-2986-4eba-b47d-dd1f4b83e28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:18.290103 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 994/1964] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:18 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:51:18.512752 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7fc17-7e23-44e5-af5b-7b6f4727b104 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:18.515089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-9ac7fc17-7e23-44e5-af5b-7b6f4727b104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:18.575747 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac7fc17-7e23-44e5-af5b-7b6f4727b104 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:18.579609 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a536df-72ff-4477-8b1d-e0e9f5312368 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:18.580377 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a536df-72ff-4477-8b1d-e0e9f5312368 service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:18.580534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a536df-72ff-4477-8b1d-e0e9f5312368 service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:18.580809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a536df-72ff-4477-8b1d-e0e9f5312368 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:18.591616 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a536df-72ff-4477-8b1d-e0e9f5312368 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:18.593191 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a536df-72ff-4477-8b1d-e0e9f5312368 service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:18.606620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 971/1965] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:18 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:19.165221 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d196048a-2986-4eba-b47d-dd1f4b83e28a None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:19.166671 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d196048a-2986-4eba-b47d-dd1f4b83e28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:19.178500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d196048a-2986-4eba-b47d-dd1f4b83e28a None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:19.182951 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbc7bed-d1e3-4e68-a38d-66cd613068ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:19.183636 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc7bed-d1e3-4e68-a38d-66cd613068ce service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:19.183923 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc7bed-d1e3-4e68-a38d-66cd613068ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:19.184202 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbc7bed-d1e3-4e68-a38d-66cd613068ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:19.244808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc7bed-d1e3-4e68-a38d-66cd613068ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:19.246305 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbc7bed-d1e3-4e68-a38d-66cd613068ce service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:19.255894 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 995/1966] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:19 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:22.257141 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a536df-72ff-4477-8b1d-e0e9f5312368 service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:22.258580 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-92a536df-72ff-4477-8b1d-e0e9f5312368 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:22.304172 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:22.331615 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a536df-72ff-4477-8b1d-e0e9f5312368 service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:22.336161 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c29504-d7b0-40e3-ad02-4f42f84bcdf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:22.337591 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c29504-d7b0-40e3-ad02-4f42f84bcdf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:22.337894 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c29504-d7b0-40e3-ad02-4f42f84bcdf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:22.338149 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c29504-d7b0-40e3-ad02-4f42f84bcdf1 admin admin] PATH_INFO: `/v3/users/5b66b3e0d7ea43079df33888f68b9fd6` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:22.339350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c29504-d7b0-40e3-ad02-4f42f84bcdf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b66b3e0d7ea43079df33888f68b9fd6)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:22.341916 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c29504-d7b0-40e3-ad02-4f42f84bcdf1 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:22.550645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-30c29504-d7b0-40e3-ad02-4f42f84bcdf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b66b3e0d7ea43079df33888f68b9fd6'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:22.565311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-30c29504-d7b0-40e3-ad02-4f42f84bcdf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b66b3e0d7ea43079df33888f68b9fd6'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:22.673391 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-30c29504-d7b0-40e3-ad02-4f42f84bcdf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b66b3e0d7ea43079df33888f68b9fd6'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:22.677051 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-30c29504-d7b0-40e3-ad02-4f42f84bcdf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b66b3e0d7ea43079df33888f68b9fd6'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:22.677051 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-30c29504-d7b0-40e3-ad02-4f42f84bcdf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b66b3e0d7ea43079df33888f68b9fd6'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:22.691069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 972/1967] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:22 2022] DELETE /identity/v3/users/5b66b3e0d7ea43079df33888f68b9fd6 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:22.696332 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d08b63-4809-4cdc-a996-72eafca23873 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:22.696941 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d08b63-4809-4cdc-a996-72eafca23873 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:22.697079 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d08b63-4809-4cdc-a996-72eafca23873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:22.724199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-c8d08b63-4809-4cdc-a996-72eafca23873 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:22.725671 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-c8d08b63-4809-4cdc-a996-72eafca23873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:22.765565 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:22.797023 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 996/1968] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:22.813596 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c29504-d7b0-40e3-ad02-4f42f84bcdf1 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:22.815196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-30c29504-d7b0-40e3-ad02-4f42f84bcdf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:22.887977 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c29504-d7b0-40e3-ad02-4f42f84bcdf1 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:22.891593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256a42a-717f-46df-8067-6c3ae979bf74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:22.892244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256a42a-717f-46df-8067-6c3ae979bf74 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:22.892527 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256a42a-717f-46df-8067-6c3ae979bf74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:22.892861 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4256a42a-717f-46df-8067-6c3ae979bf74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:22.903457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4256a42a-717f-46df-8067-6c3ae979bf74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:22.905210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4256a42a-717f-46df-8067-6c3ae979bf74 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:22.916514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 973/1969] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:23.547739 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d08b63-4809-4cdc-a996-72eafca23873 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:23.549362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-c8d08b63-4809-4cdc-a996-72eafca23873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:23.563297 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d08b63-4809-4cdc-a996-72eafca23873 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:23.568741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56768104-fe48-4cf9-8e81-3b8f5bb4870d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:23.569777 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56768104-fe48-4cf9-8e81-3b8f5bb4870d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:23.570193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56768104-fe48-4cf9-8e81-3b8f5bb4870d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:23.570577 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56768104-fe48-4cf9-8e81-3b8f5bb4870d admin admin] PATH_INFO: `/v3/projects/340927ea7f5741ea8da66fa6fc237a32` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:23.572595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56768104-fe48-4cf9-8e81-3b8f5bb4870d admin admin] RBAC: Authorizing `identity:delete_project(project_id=340927ea7f5741ea8da66fa6fc237a32)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:23.574847 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56768104-fe48-4cf9-8e81-3b8f5bb4870d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:23.698895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-56768104-fe48-4cf9-8e81-3b8f5bb4870d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '340927ea7f5741ea8da66fa6fc237a32', 'request_id': 'req-56768104-fe48-4cf9-8e81-3b8f5bb4870d'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:23.705553 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-56768104-fe48-4cf9-8e81-3b8f5bb4870d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '340927ea7f5741ea8da66fa6fc237a32', 'request_id': 'req-56768104-fe48-4cf9-8e81-3b8f5bb4870d'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:23.712447 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-56768104-fe48-4cf9-8e81-3b8f5bb4870d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '340927ea7f5741ea8da66fa6fc237a32', 'request_id': 'req-56768104-fe48-4cf9-8e81-3b8f5bb4870d'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:23.799549 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256a42a-717f-46df-8067-6c3ae979bf74 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:23.801160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-4256a42a-717f-46df-8067-6c3ae979bf74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:23.815248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4256a42a-717f-46df-8067-6c3ae979bf74 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:23.819482 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-56768104-fe48-4cf9-8e81-3b8f5bb4870d admin admin] The token cache is being invalidate because project 340927ea7f5741ea8da66fa6fc237a32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:51:23.819770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-56768104-fe48-4cf9-8e81-3b8f5bb4870d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:23.820544 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-56768104-fe48-4cf9-8e81-3b8f5bb4870d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:23.822471 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 997/1970] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:23 2022] DELETE /identity/v3/projects/340927ea7f5741ea8da66fa6fc237a32 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:23.826435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56768104-fe48-4cf9-8e81-3b8f5bb4870d admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:23.827980 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-56768104-fe48-4cf9-8e81-3b8f5bb4870d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:23.865798 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:23.874790 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea065b-fc72-407d-a5bc-a71e7695e46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:23.875715 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea065b-fc72-407d-a5bc-a71e7695e46a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:23.875993 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea065b-fc72-407d-a5bc-a71e7695e46a admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:23.876267 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eea065b-fc72-407d-a5bc-a71e7695e46a admin admin] PATH_INFO: `/v3/users/a819d84211dc4cc7a0d21664378e7b70` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:23.877483 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea065b-fc72-407d-a5bc-a71e7695e46a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a819d84211dc4cc7a0d21664378e7b70)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:23.879406 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eea065b-fc72-407d-a5bc-a71e7695e46a admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:23.897669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56768104-fe48-4cf9-8e81-3b8f5bb4870d admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:23.903269 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0858b-eee5-4728-8f1a-56b947bdcf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:23.904144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0858b-eee5-4728-8f1a-56b947bdcf42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:23.904428 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0858b-eee5-4728-8f1a-56b947bdcf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:23.904704 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a0858b-eee5-4728-8f1a-56b947bdcf42 admin admin] PATH_INFO: `/v3/users/d73c96ae7b444d8287b96b0f24f81d47` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:23.906005 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a0858b-eee5-4728-8f1a-56b947bdcf42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d73c96ae7b444d8287b96b0f24f81d47)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:23.907959 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a0858b-eee5-4728-8f1a-56b947bdcf42 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:24.198811 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d7a0858b-eee5-4728-8f1a-56b947bdcf42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd73c96ae7b444d8287b96b0f24f81d47'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:24.222576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d7a0858b-eee5-4728-8f1a-56b947bdcf42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd73c96ae7b444d8287b96b0f24f81d47'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:24.225238 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:24.229216 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1eea065b-fc72-407d-a5bc-a71e7695e46a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a819d84211dc4cc7a0d21664378e7b70'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:24.243574 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1eea065b-fc72-407d-a5bc-a71e7695e46a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a819d84211dc4cc7a0d21664378e7b70'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:24.323847 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d7a0858b-eee5-4728-8f1a-56b947bdcf42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd73c96ae7b444d8287b96b0f24f81d47'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:24.324467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d7a0858b-eee5-4728-8f1a-56b947bdcf42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd73c96ae7b444d8287b96b0f24f81d47'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:24.340442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d7a0858b-eee5-4728-8f1a-56b947bdcf42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd73c96ae7b444d8287b96b0f24f81d47'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:24.340442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 998/1971] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:23 2022] DELETE /identity/v3/users/d73c96ae7b444d8287b96b0f24f81d47 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:51:24.392902 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1eea065b-fc72-407d-a5bc-a71e7695e46a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a819d84211dc4cc7a0d21664378e7b70'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:24.393622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1eea065b-fc72-407d-a5bc-a71e7695e46a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a819d84211dc4cc7a0d21664378e7b70'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:24.394237 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1eea065b-fc72-407d-a5bc-a71e7695e46a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a819d84211dc4cc7a0d21664378e7b70'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:24.407864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 974/1972] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:23 2022] DELETE /identity/v3/users/a819d84211dc4cc7a0d21664378e7b70 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:51:24.668338 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0858b-eee5-4728-8f1a-56b947bdcf42 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:24.670043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d7a0858b-eee5-4728-8f1a-56b947bdcf42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:24.742498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a0858b-eee5-4728-8f1a-56b947bdcf42 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:24.777807 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea065b-fc72-407d-a5bc-a71e7695e46a admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:24.779482 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-1eea065b-fc72-407d-a5bc-a71e7695e46a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:24.811607 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98c94b-1a33-451a-9913-60b6954c88eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:24.812608 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98c94b-1a33-451a-9913-60b6954c88eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:24.812990 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98c94b-1a33-451a-9913-60b6954c88eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:24.813353 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98c94b-1a33-451a-9913-60b6954c88eb admin admin] PATH_INFO: `/v3/projects/f22a0940d9a24f4e940313e1f5e55e8d` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:24.814652 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98c94b-1a33-451a-9913-60b6954c88eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f22a0940d9a24f4e940313e1f5e55e8d)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:24.816624 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98c94b-1a33-451a-9913-60b6954c88eb admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:24.859346 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eea065b-fc72-407d-a5bc-a71e7695e46a admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:24.864034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615ed763-fab1-4bb1-95bd-856a3f09dee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:24.864912 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ed763-fab1-4bb1-95bd-856a3f09dee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:24.865227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ed763-fab1-4bb1-95bd-856a3f09dee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:24.865543 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ed763-fab1-4bb1-95bd-856a3f09dee5 admin admin] PATH_INFO: `/v3/projects/2fe434ee7dc74190b66e1875bfa986e8` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:24.866739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ed763-fab1-4bb1-95bd-856a3f09dee5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fe434ee7dc74190b66e1875bfa986e8)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:24.868765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615ed763-fab1-4bb1-95bd-856a3f09dee5 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:24.942217 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-ba98c94b-1a33-451a-9913-60b6954c88eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f22a0940d9a24f4e940313e1f5e55e8d', 'request_id': 'req-ba98c94b-1a33-451a-9913-60b6954c88eb'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:24.948451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-ba98c94b-1a33-451a-9913-60b6954c88eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f22a0940d9a24f4e940313e1f5e55e8d', 'request_id': 'req-ba98c94b-1a33-451a-9913-60b6954c88eb'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:24.954669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-ba98c94b-1a33-451a-9913-60b6954c88eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f22a0940d9a24f4e940313e1f5e55e8d', 'request_id': 'req-ba98c94b-1a33-451a-9913-60b6954c88eb'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:24.999441 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-615ed763-fab1-4bb1-95bd-856a3f09dee5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fe434ee7dc74190b66e1875bfa986e8', 'request_id': 'req-615ed763-fab1-4bb1-95bd-856a3f09dee5'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:25.004872 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-615ed763-fab1-4bb1-95bd-856a3f09dee5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fe434ee7dc74190b66e1875bfa986e8', 'request_id': 'req-615ed763-fab1-4bb1-95bd-856a3f09dee5'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:25.055714 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-ba98c94b-1a33-451a-9913-60b6954c88eb admin admin] The token cache is being invalidate because project f22a0940d9a24f4e940313e1f5e55e8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:51:25.056152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-ba98c94b-1a33-451a-9913-60b6954c88eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:25.056783 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-ba98c94b-1a33-451a-9913-60b6954c88eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:25.058361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 999/1973] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:24 2022] DELETE /identity/v3/projects/f22a0940d9a24f4e940313e1f5e55e8d => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:25.062213 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98c94b-1a33-451a-9913-60b6954c88eb admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:25.063487 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ba98c94b-1a33-451a-9913-60b6954c88eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:25.085191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:25.123767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-615ed763-fab1-4bb1-95bd-856a3f09dee5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fe434ee7dc74190b66e1875bfa986e8', 'request_id': 'req-615ed763-fab1-4bb1-95bd-856a3f09dee5'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:25.129218 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98c94b-1a33-451a-9913-60b6954c88eb admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:25.136494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-615ed763-fab1-4bb1-95bd-856a3f09dee5 admin admin] The token cache is being invalidate because project 2fe434ee7dc74190b66e1875bfa986e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:51:25.139446 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-615ed763-fab1-4bb1-95bd-856a3f09dee5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:25.139446 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-615ed763-fab1-4bb1-95bd-856a3f09dee5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:25.139446 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 975/1974] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:24 2022] DELETE /identity/v3/projects/2fe434ee7dc74190b66e1875bfa986e8 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:25.143789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ef140-026b-47e7-b40c-5bd5d1452fce None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:25.144152 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ef140-026b-47e7-b40c-5bd5d1452fce None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:25.144500 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63ef140-026b-47e7-b40c-5bd5d1452fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:25.152739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:25.164453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-b63ef140-026b-47e7-b40c-5bd5d1452fce None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:25.166683 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-b63ef140-026b-47e7-b40c-5bd5d1452fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:25.197412 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6663430-21a5-4f99-90ff-f67fd732bb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:25.198392 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6663430-21a5-4f99-90ff-f67fd732bb2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:25.198676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6663430-21a5-4f99-90ff-f67fd732bb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:25.198953 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6663430-21a5-4f99-90ff-f67fd732bb2f admin admin] PATH_INFO: `/v3/users/5de20f1bd36e4137bc3d010357f06b26` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:25.200182 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6663430-21a5-4f99-90ff-f67fd732bb2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5de20f1bd36e4137bc3d010357f06b26)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:25.202240 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6663430-21a5-4f99-90ff-f67fd732bb2f admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:25.234953 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 976/1975] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:25.242408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11880836-8392-4aa4-a9b7-97e43361c718 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:25.242700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11880836-8392-4aa4-a9b7-97e43361c718 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:25.243502 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11880836-8392-4aa4-a9b7-97e43361c718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:25.262249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-11880836-8392-4aa4-a9b7-97e43361c718 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:25.263438 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-11880836-8392-4aa4-a9b7-97e43361c718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:25.271434 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 977/1976] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:51:25.275536 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11880836-8392-4aa4-a9b7-97e43361c718 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:25.285074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11880836-8392-4aa4-a9b7-97e43361c718 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:25.289790 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85d762-3764-4c50-8695-e28334c5db1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:25.290466 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85d762-3764-4c50-8695-e28334c5db1a admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:25.290777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85d762-3764-4c50-8695-e28334c5db1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:25.291057 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85d762-3764-4c50-8695-e28334c5db1a admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:25.291812 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e85d762-3764-4c50-8695-e28334c5db1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:51:25.292121 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e85d762-3764-4c50-8695-e28334c5db1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:25.293762 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e85d762-3764-4c50-8695-e28334c5db1a admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:25.306490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 978/1977] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:51:25.310677 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85d762-3764-4c50-8695-e28334c5db1a admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:25.323157 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85d762-3764-4c50-8695-e28334c5db1a admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:25.329280 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a9732-1658-4cd6-ae61-e4367cfff23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:25.330011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a9732-1658-4cd6-ae61-e4367cfff23b admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:25.330355 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a9732-1658-4cd6-ae61-e4367cfff23b admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:25.330696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a9732-1658-4cd6-ae61-e4367cfff23b admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:25.333265 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a9732-1658-4cd6-ae61-e4367cfff23b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:25.333735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a9732-1658-4cd6-ae61-e4367cfff23b admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:25.418294 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f6663430-21a5-4f99-90ff-f67fd732bb2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5de20f1bd36e4137bc3d010357f06b26'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:25.432560 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f6663430-21a5-4f99-90ff-f67fd732bb2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5de20f1bd36e4137bc3d010357f06b26'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:25.436849 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 979/1978] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:51:25 2022] POST /identity/v3/projects => generated 379 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:51:25.440725 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a9732-1658-4cd6-ae61-e4367cfff23b admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:25.499597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f6663430-21a5-4f99-90ff-f67fd732bb2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5de20f1bd36e4137bc3d010357f06b26'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:25.500223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f6663430-21a5-4f99-90ff-f67fd732bb2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5de20f1bd36e4137bc3d010357f06b26'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:25.504752 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a9732-1658-4cd6-ae61-e4367cfff23b admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:25.508045 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f6663430-21a5-4f99-90ff-f67fd732bb2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5de20f1bd36e4137bc3d010357f06b26'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:25.511315 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61725980-cfea-4be4-86c0-67c635829780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:25.512084 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61725980-cfea-4be4-86c0-67c635829780 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:25.512441 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61725980-cfea-4be4-86c0-67c635829780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:25.512782 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61725980-cfea-4be4-86c0-67c635829780 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:25.513578 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61725980-cfea-4be4-86c0-67c635829780 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:25.515508 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1000/1979] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:25 2022] DELETE /identity/v3/users/5de20f1bd36e4137bc3d010357f06b26 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:51:25.515664 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61725980-cfea-4be4-86c0-67c635829780 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:25.626942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 980/1980] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:51:25 2022] POST /identity/v3/users => generated 290 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:51:25.630788 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6663430-21a5-4f99-90ff-f67fd732bb2f admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:25.632224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-f6663430-21a5-4f99-90ff-f67fd732bb2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:25.698717 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6663430-21a5-4f99-90ff-f67fd732bb2f admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:25.703333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e2a40-d016-41b7-a102-6d62140c6ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:25.704000 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e2a40-d016-41b7-a102-6d62140c6ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:25.704287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e2a40-d016-41b7-a102-6d62140c6ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:25.704572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e2a40-d016-41b7-a102-6d62140c6ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:25.705341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e2a40-d016-41b7-a102-6d62140c6ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:25.706967 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e2a40-d016-41b7-a102-6d62140c6ee3 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:25.718202 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1001/1981] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:25.722600 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61725980-cfea-4be4-86c0-67c635829780 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:25.733104 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61725980-cfea-4be4-86c0-67c635829780 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:25.737810 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a99251-2ef0-46d2-a86f-434a9b3bc34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:25.738366 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a99251-2ef0-46d2-a86f-434a9b3bc34e admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:25.744634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a99251-2ef0-46d2-a86f-434a9b3bc34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:25.744634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a99251-2ef0-46d2-a86f-434a9b3bc34e admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:25.744634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a99251-2ef0-46d2-a86f-434a9b3bc34e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:25.744634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a99251-2ef0-46d2-a86f-434a9b3bc34e admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:25.747513 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e2a40-d016-41b7-a102-6d62140c6ee3 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:25.749458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ad3e2a40-d016-41b7-a102-6d62140c6ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:25.755279 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 981/1982] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:25.756148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a99251-2ef0-46d2-a86f-434a9b3bc34e admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:25.764481 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e2a40-d016-41b7-a102-6d62140c6ee3 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:25.767841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a99251-2ef0-46d2-a86f-434a9b3bc34e admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:25.769871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b449d8c-1898-4c8b-aa4b-b38bde48a9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:25.770913 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b449d8c-1898-4c8b-aa4b-b38bde48a9ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:25.771263 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b449d8c-1898-4c8b-aa4b-b38bde48a9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:25.771474 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b449d8c-1898-4c8b-aa4b-b38bde48a9ea admin admin] PATH_INFO: `/v3/projects/5ab2f42bffd94622bf35b18e160a47c9` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:25.772726 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b449d8c-1898-4c8b-aa4b-b38bde48a9ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ab2f42bffd94622bf35b18e160a47c9)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:25.775979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9790e47-d009-40a8-8dce-550a4108b34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:25.777138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9790e47-d009-40a8-8dce-550a4108b34b admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:25.777677 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9790e47-d009-40a8-8dce-550a4108b34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:25.777677 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9790e47-d009-40a8-8dce-550a4108b34b admin admin] PATH_INFO: `/v3/projects/851046691d294b1592f08f509bfedbbc/users/13ddfaf9ad514026bfb589a2ba494e27/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:25.778225 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b449d8c-1898-4c8b-aa4b-b38bde48a9ea admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:25.799604 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9790e47-d009-40a8-8dce-550a4108b34b admin admin] RBAC: Authorizing `identity:create_grant(project_id=851046691d294b1592f08f509bfedbbc, user_id=13ddfaf9ad514026bfb589a2ba494e27, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:25.803450 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9790e47-d009-40a8-8dce-550a4108b34b admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:25.913425 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 982/1983] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:25 2022] PUT /identity/v3/projects/851046691d294b1592f08f509bfedbbc/users/13ddfaf9ad514026bfb589a2ba494e27/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:25.925296 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983c9f7-241a-4bc6-b200-e41176b21fe5 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:25.925587 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983c9f7-241a-4bc6-b200-e41176b21fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:25.926033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2983c9f7-241a-4bc6-b200-e41176b21fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:25.933360 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-8b449d8c-1898-4c8b-aa4b-b38bde48a9ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ab2f42bffd94622bf35b18e160a47c9', 'request_id': 'req-8b449d8c-1898-4c8b-aa4b-b38bde48a9ea'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:25.938079 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:25.944546 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-8b449d8c-1898-4c8b-aa4b-b38bde48a9ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ab2f42bffd94622bf35b18e160a47c9', 'request_id': 'req-8b449d8c-1898-4c8b-aa4b-b38bde48a9ea'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:25.953898 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-8b449d8c-1898-4c8b-aa4b-b38bde48a9ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ab2f42bffd94622bf35b18e160a47c9', 'request_id': 'req-8b449d8c-1898-4c8b-aa4b-b38bde48a9ea'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:25.965334 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-2983c9f7-241a-4bc6-b200-e41176b21fe5 None None] MFA Rules not processed for user `13ddfaf9ad514026bfb589a2ba494e27`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:25.966587 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-2983c9f7-241a-4bc6-b200-e41176b21fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:26.031554 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:26.037065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 983/1984] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:51:26.041904 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983c9f7-241a-4bc6-b200-e41176b21fe5 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:26.076230 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-8b449d8c-1898-4c8b-aa4b-b38bde48a9ea admin admin] The token cache is being invalidate because project 5ab2f42bffd94622bf35b18e160a47c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:51:26.077740 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-8b449d8c-1898-4c8b-aa4b-b38bde48a9ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:26.078874 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-8b449d8c-1898-4c8b-aa4b-b38bde48a9ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:26.085204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1002/1985] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:25 2022] DELETE /identity/v3/projects/5ab2f42bffd94622bf35b18e160a47c9 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:51:26.087640 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c578c21-72cb-441a-a029-92cd6e4fb7c3 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:26.089425 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c578c21-72cb-441a-a029-92cd6e4fb7c3 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:26.089425 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c578c21-72cb-441a-a029-92cd6e4fb7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:26.115246 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-1c578c21-72cb-441a-a029-92cd6e4fb7c3 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:26.116211 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2983c9f7-241a-4bc6-b200-e41176b21fe5 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:26.116774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-1c578c21-72cb-441a-a029-92cd6e4fb7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:26.132395 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573655b-6272-44f6-8b55-dea753208674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:26.132395 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573655b-6272-44f6-8b55-dea753208674 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:26.132395 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573655b-6272-44f6-8b55-dea753208674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:26.132395 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573655b-6272-44f6-8b55-dea753208674 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:26.132743 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a573655b-6272-44f6-8b55-dea753208674 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:26.132743 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a573655b-6272-44f6-8b55-dea753208674 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:26.132896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1003/1986] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:26.134310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68df4c0-f641-472a-951a-02adfcc23bca None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:26.134822 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68df4c0-f641-472a-951a-02adfcc23bca None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:26.135168 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68df4c0-f641-472a-951a-02adfcc23bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:26.156261 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-e68df4c0-f641-472a-951a-02adfcc23bca None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:26.163009 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-e68df4c0-f641-472a-951a-02adfcc23bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:26.168050 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1004/1987] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:26.172584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68df4c0-f641-472a-951a-02adfcc23bca None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:26.182194 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68df4c0-f641-472a-951a-02adfcc23bca None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:26.188805 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c6afd-9c37-481e-a60d-2edbaa061c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:26.189635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c6afd-9c37-481e-a60d-2edbaa061c76 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:26.190049 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c6afd-9c37-481e-a60d-2edbaa061c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:26.190421 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c6afd-9c37-481e-a60d-2edbaa061c76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:26.191482 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c6afd-9c37-481e-a60d-2edbaa061c76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:51:26.191990 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c6afd-9c37-481e-a60d-2edbaa061c76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:26.193965 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c6afd-9c37-481e-a60d-2edbaa061c76 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:26.207059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1005/1988] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:51:26.214297 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c6afd-9c37-481e-a60d-2edbaa061c76 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:26.226677 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c6afd-9c37-481e-a60d-2edbaa061c76 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:26.234780 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a210b1b-9d5f-4834-b047-1827863f2f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:26.235754 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a210b1b-9d5f-4834-b047-1827863f2f3d admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:26.237526 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a210b1b-9d5f-4834-b047-1827863f2f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:26.237526 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a210b1b-9d5f-4834-b047-1827863f2f3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:26.237526 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a210b1b-9d5f-4834-b047-1827863f2f3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:26.252898 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a210b1b-9d5f-4834-b047-1827863f2f3d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:26.253107 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 984/1989] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:51:26 2022] POST /identity/v3/projects => generated 381 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:51:26.255931 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573655b-6272-44f6-8b55-dea753208674 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:26.257777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-a573655b-6272-44f6-8b55-dea753208674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:26.335381 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573655b-6272-44f6-8b55-dea753208674 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:26.341001 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1bb0b-3213-480f-80d9-ff7d53036a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:26.341687 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1bb0b-3213-480f-80d9-ff7d53036a96 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:26.341978 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1bb0b-3213-480f-80d9-ff7d53036a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:26.342408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1bb0b-3213-480f-80d9-ff7d53036a96 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:26.343049 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1bb0b-3213-480f-80d9-ff7d53036a96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:26.345355 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1bb0b-3213-480f-80d9-ff7d53036a96 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:26.349041 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1006/1990] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:51:26 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:26.354209 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a210b1b-9d5f-4834-b047-1827863f2f3d admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:26.418516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a210b1b-9d5f-4834-b047-1827863f2f3d admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:26.423204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ddaad-3e0f-4ad3-9d29-9b8f18eb3d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:26.423883 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ddaad-3e0f-4ad3-9d29-9b8f18eb3d5b admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:26.424455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ddaad-3e0f-4ad3-9d29-9b8f18eb3d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:26.424757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ddaad-3e0f-4ad3-9d29-9b8f18eb3d5b admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:26.425693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ddaad-3e0f-4ad3-9d29-9b8f18eb3d5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:26.427431 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60ddaad-3e0f-4ad3-9d29-9b8f18eb3d5b admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:26.471231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 985/1991] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:51:26 2022] POST /identity/v3/users => generated 297 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:51:26.476836 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1bb0b-3213-480f-80d9-ff7d53036a96 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:26.497449 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1bb0b-3213-480f-80d9-ff7d53036a96 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:26.517759 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3a987-68f3-46aa-acf2-515277066ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:26.518526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3a987-68f3-46aa-acf2-515277066ec1 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:26.518883 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3a987-68f3-46aa-acf2-515277066ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:26.519281 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b3a987-68f3-46aa-acf2-515277066ec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:26.520235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3a987-68f3-46aa-acf2-515277066ec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:26.522086 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b3a987-68f3-46aa-acf2-515277066ec1 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:26.533504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1007/1992] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:51:26 2022] POST /identity/v3/users => generated 292 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:26.537402 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ddaad-3e0f-4ad3-9d29-9b8f18eb3d5b admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:26.549828 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 986/1993] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:26.553655 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3a987-68f3-46aa-acf2-515277066ec1 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:26.562319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60ddaad-3e0f-4ad3-9d29-9b8f18eb3d5b admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:26.577931 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30001b-bf12-40d0-9d38-9d46291bfacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:26.578068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b3a987-68f3-46aa-acf2-515277066ec1 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:26.580941 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30001b-bf12-40d0-9d38-9d46291bfacf admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:26.581361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30001b-bf12-40d0-9d38-9d46291bfacf admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:26.581749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30001b-bf12-40d0-9d38-9d46291bfacf admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:26.582599 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf30001b-bf12-40d0-9d38-9d46291bfacf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:26.584329 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf30001b-bf12-40d0-9d38-9d46291bfacf admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:26.587939 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fb388-0f2f-430e-a806-a5ff569ecb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:26.588861 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fb388-0f2f-430e-a806-a5ff569ecb6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:26.589636 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fb388-0f2f-430e-a806-a5ff569ecb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:26.589636 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fb388-0f2f-430e-a806-a5ff569ecb6a admin admin] PATH_INFO: `/v3/projects/798ce34345de4bf2b12d41b4dc6c6e53/users/08b0b01529924b469f0be9fa3047cad7/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:26.596237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:26.597877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:26.625821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1008/1994] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:51:26.626020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fb388-0f2f-430e-a806-a5ff569ecb6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=798ce34345de4bf2b12d41b4dc6c6e53, user_id=08b0b01529924b469f0be9fa3047cad7, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:26.627698 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30001b-bf12-40d0-9d38-9d46291bfacf admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:26.627844 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7fb388-0f2f-430e-a806-a5ff569ecb6a admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:26.642534 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30001b-bf12-40d0-9d38-9d46291bfacf admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:26.648428 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618ee0a-1ebe-4bd0-9792-8e04f03f9ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:26.655025 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618ee0a-1ebe-4bd0-9792-8e04f03f9ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:26.655420 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618ee0a-1ebe-4bd0-9792-8e04f03f9ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:26.655814 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618ee0a-1ebe-4bd0-9792-8e04f03f9ae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:26.657455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d618ee0a-1ebe-4bd0-9792-8e04f03f9ae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:26.659909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d618ee0a-1ebe-4bd0-9792-8e04f03f9ae2 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:26.687494 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1009/1995] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:26.688878 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618ee0a-1ebe-4bd0-9792-8e04f03f9ae2 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:26.705621 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618ee0a-1ebe-4bd0-9792-8e04f03f9ae2 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:26.710516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11532c-0715-44e7-9556-1ea7a8a05d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:26.714433 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11532c-0715-44e7-9556-1ea7a8a05d91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:26.714433 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11532c-0715-44e7-9556-1ea7a8a05d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:26.714433 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11532c-0715-44e7-9556-1ea7a8a05d91 admin admin] PATH_INFO: `/v3/projects/04eb80484f754d6fb5155d374929361f/users/31cc1bab33e44260a48fd34802508f41/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:26.727321 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11532c-0715-44e7-9556-1ea7a8a05d91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04eb80484f754d6fb5155d374929361f, user_id=31cc1bab33e44260a48fd34802508f41, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:26.730106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11532c-0715-44e7-9556-1ea7a8a05d91 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:26.741626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 987/1996] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:26 2022] PUT /identity/v3/projects/798ce34345de4bf2b12d41b4dc6c6e53/users/08b0b01529924b469f0be9fa3047cad7/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:26.751705 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fb388-0f2f-430e-a806-a5ff569ecb6a admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:26.814531 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fb388-0f2f-430e-a806-a5ff569ecb6a admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:26.819239 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd16b9-c271-48b0-b033-d1461f2b64f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:26.820029 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd16b9-c271-48b0-b033-d1461f2b64f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:26.820309 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd16b9-c271-48b0-b033-d1461f2b64f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:26.820585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd16b9-c271-48b0-b033-d1461f2b64f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:26.821900 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd16b9-c271-48b0-b033-d1461f2b64f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:26.823498 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd16b9-c271-48b0-b033-d1461f2b64f0 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:26.845824 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1010/1997] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:26 2022] PUT /identity/v3/projects/04eb80484f754d6fb5155d374929361f/users/31cc1bab33e44260a48fd34802508f41/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:26.846273 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 988/1998] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:26 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:51:26.846394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11532c-0715-44e7-9556-1ea7a8a05d91 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:26.855377 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434202a-2a12-4f6b-ad3b-18a0cd1ff92f None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:26.855673 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434202a-2a12-4f6b-ad3b-18a0cd1ff92f None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:26.855960 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434202a-2a12-4f6b-ad3b-18a0cd1ff92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:26.891865 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-6434202a-2a12-4f6b-ad3b-18a0cd1ff92f None None] MFA Rules not processed for user `31cc1bab33e44260a48fd34802508f41`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:26.893224 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-6434202a-2a12-4f6b-ad3b-18a0cd1ff92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:26.952460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11532c-0715-44e7-9556-1ea7a8a05d91 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:26.955259 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1d10cb-56da-44a7-89a1-9fc5259bb3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:26.956278 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d10cb-56da-44a7-89a1-9fc5259bb3af admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:26.956701 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d10cb-56da-44a7-89a1-9fc5259bb3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:26.957696 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1d10cb-56da-44a7-89a1-9fc5259bb3af admin admin] PATH_INFO: `/v3/projects/798ce34345de4bf2b12d41b4dc6c6e53/users/08b0b01529924b469f0be9fa3047cad7/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:26.959815 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d10cb-56da-44a7-89a1-9fc5259bb3af admin admin] RBAC: Authorizing `identity:create_grant(project_id=798ce34345de4bf2b12d41b4dc6c6e53, user_id=08b0b01529924b469f0be9fa3047cad7, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:26.962888 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1d10cb-56da-44a7-89a1-9fc5259bb3af admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:26.976938 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1011/1999] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:26 2022] PUT /identity/v3/projects/798ce34345de4bf2b12d41b4dc6c6e53/users/08b0b01529924b469f0be9fa3047cad7/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:26.977196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 989/2000] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:26.987981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31316e8-2a3b-4567-8c6c-d2247b9cb99d None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:26.988299 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31316e8-2a3b-4567-8c6c-d2247b9cb99d None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:26.988624 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31316e8-2a3b-4567-8c6c-d2247b9cb99d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:27.016256 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-f31316e8-2a3b-4567-8c6c-d2247b9cb99d None None] MFA Rules not processed for user `08b0b01529924b469f0be9fa3047cad7`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:27.017492 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-f31316e8-2a3b-4567-8c6c-d2247b9cb99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:27.092403 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1012/2001] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:26 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:27.227202 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434202a-2a12-4f6b-ad3b-18a0cd1ff92f None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:27.229258 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-6434202a-2a12-4f6b-ad3b-18a0cd1ff92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:27.286258 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434202a-2a12-4f6b-ad3b-18a0cd1ff92f None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:27.313839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:27.318774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31316e8-2a3b-4567-8c6c-d2247b9cb99d None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:27.320194 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-f31316e8-2a3b-4567-8c6c-d2247b9cb99d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:27.341682 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63fc53-69fc-4afc-8371-d85ef5911376 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:27.342455 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63fc53-69fc-4afc-8371-d85ef5911376 service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:27.342858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63fc53-69fc-4afc-8371-d85ef5911376 service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:27.343142 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63fc53-69fc-4afc-8371-d85ef5911376 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:27.365989 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:27.381643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31316e8-2a3b-4567-8c6c-d2247b9cb99d None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:27.386055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1430062-c18b-4c25-96ae-fb78b35a1e08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:27.386708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1430062-c18b-4c25-96ae-fb78b35a1e08 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:27.386988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1430062-c18b-4c25-96ae-fb78b35a1e08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:27.387403 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1430062-c18b-4c25-96ae-fb78b35a1e08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:27.398650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1430062-c18b-4c25-96ae-fb78b35a1e08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:27.401742 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1430062-c18b-4c25-96ae-fb78b35a1e08 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:27.404620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63fc53-69fc-4afc-8371-d85ef5911376 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:27.406103 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63fc53-69fc-4afc-8371-d85ef5911376 service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:27.412637 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1013/2002] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:27.417497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 990/2003] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:27.436611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1430062-c18b-4c25-96ae-fb78b35a1e08 service cinder] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:27.513458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1430062-c18b-4c25-96ae-fb78b35a1e08 service cinder] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:27.519278 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5d5cb-19b9-4732-af63-c9cc132c1efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:27.520873 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5d5cb-19b9-4732-af63-c9cc132c1efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:27.521284 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5d5cb-19b9-4732-af63-c9cc132c1efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:27.521651 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5d5cb-19b9-4732-af63-c9cc132c1efe admin admin] PATH_INFO: `/v3/users/31cc1bab33e44260a48fd34802508f41` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:27.527313 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5d5cb-19b9-4732-af63-c9cc132c1efe admin admin] RBAC: Authorizing `identity:delete_user(user_id=31cc1bab33e44260a48fd34802508f41)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:27.529429 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5d5cb-19b9-4732-af63-c9cc132c1efe admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:27.638703 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63fc53-69fc-4afc-8371-d85ef5911376 service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:27.647124 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63fc53-69fc-4afc-8371-d85ef5911376 service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:27.651467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e7452-5274-42ad-bc8c-9e3310258b9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:27.652111 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e7452-5274-42ad-bc8c-9e3310258b9a service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:27.652396 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e7452-5274-42ad-bc8c-9e3310258b9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:27.652676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e7452-5274-42ad-bc8c-9e3310258b9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:27.654483 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-bf8e7452-5274-42ad-bc8c-9e3310258b9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:27.715844 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e7452-5274-42ad-bc8c-9e3310258b9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:27.717353 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8e7452-5274-42ad-bc8c-9e3310258b9a service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:27.726802 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 991/2004] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:51:27.728548 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e7452-5274-42ad-bc8c-9e3310258b9a service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:27.729965 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-bf8e7452-5274-42ad-bc8c-9e3310258b9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:27.743928 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8e7452-5274-42ad-bc8c-9e3310258b9a service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:27.751100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84892f-a0e6-4fcb-bb6c-8c0d73e182b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:27.755534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84892f-a0e6-4fcb-bb6c-8c0d73e182b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:27.755534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84892f-a0e6-4fcb-bb6c-8c0d73e182b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:27.755534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c84892f-a0e6-4fcb-bb6c-8c0d73e182b5 admin admin] PATH_INFO: `/v3/users/181f4390891b478a840be24f469cf468` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:27.755534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84892f-a0e6-4fcb-bb6c-8c0d73e182b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=181f4390891b478a840be24f469cf468)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:27.756597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c84892f-a0e6-4fcb-bb6c-8c0d73e182b5 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:27.799751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-81d5d5cb-19b9-4732-af63-c9cc132c1efe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31cc1bab33e44260a48fd34802508f41'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:27.813721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-81d5d5cb-19b9-4732-af63-c9cc132c1efe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31cc1bab33e44260a48fd34802508f41'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:27.924815 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-81d5d5cb-19b9-4732-af63-c9cc132c1efe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31cc1bab33e44260a48fd34802508f41'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:27.927177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-81d5d5cb-19b9-4732-af63-c9cc132c1efe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31cc1bab33e44260a48fd34802508f41'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:27.938951 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-81d5d5cb-19b9-4732-af63-c9cc132c1efe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31cc1bab33e44260a48fd34802508f41'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:27.945342 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1014/2005] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:27 2022] DELETE /identity/v3/users/31cc1bab33e44260a48fd34802508f41 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:27.950735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96640824-610d-4724-a0cb-b9b506bdf4a1 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:27.951033 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96640824-610d-4724-a0cb-b9b506bdf4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:27.951333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96640824-610d-4724-a0cb-b9b506bdf4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:27.980620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-96640824-610d-4724-a0cb-b9b506bdf4a1 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:27.981870 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-96640824-610d-4724-a0cb-b9b506bdf4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:28.044558 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1015/2006] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:28.054330 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96640824-610d-4724-a0cb-b9b506bdf4a1 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:28.055697 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-96640824-610d-4724-a0cb-b9b506bdf4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:28.061137 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:28.087592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0c84892f-a0e6-4fcb-bb6c-8c0d73e182b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '181f4390891b478a840be24f469cf468'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:28.101757 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0c84892f-a0e6-4fcb-bb6c-8c0d73e182b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '181f4390891b478a840be24f469cf468'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:28.118537 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96640824-610d-4724-a0cb-b9b506bdf4a1 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:28.122403 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee297f13-0169-47e6-aa2c-ffd4a805f63d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:28.123059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee297f13-0169-47e6-aa2c-ffd4a805f63d service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:28.123345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee297f13-0169-47e6-aa2c-ffd4a805f63d service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:28.123626 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee297f13-0169-47e6-aa2c-ffd4a805f63d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:28.132631 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee297f13-0169-47e6-aa2c-ffd4a805f63d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:28.134825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee297f13-0169-47e6-aa2c-ffd4a805f63d service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:28.144548 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1016/2007] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:51:28.168167 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0c84892f-a0e6-4fcb-bb6c-8c0d73e182b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '181f4390891b478a840be24f469cf468'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:28.177132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0c84892f-a0e6-4fcb-bb6c-8c0d73e182b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '181f4390891b478a840be24f469cf468'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:28.177132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0c84892f-a0e6-4fcb-bb6c-8c0d73e182b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '181f4390891b478a840be24f469cf468'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:28.177132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:28.210141 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 992/2008] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:27 2022] DELETE /identity/v3/users/181f4390891b478a840be24f469cf468 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:28.215185 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a567b25-9518-4b40-a5ca-d4757fd67276 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:28.215484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a567b25-9518-4b40-a5ca-d4757fd67276 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:28.215770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a567b25-9518-4b40-a5ca-d4757fd67276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:28.234815 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-1a567b25-9518-4b40-a5ca-d4757fd67276 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:28.235780 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-1a567b25-9518-4b40-a5ca-d4757fd67276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:28.295656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1017/2009] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:28.305082 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84892f-a0e6-4fcb-bb6c-8c0d73e182b5 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:28.307176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-0c84892f-a0e6-4fcb-bb6c-8c0d73e182b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:28.381390 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c84892f-a0e6-4fcb-bb6c-8c0d73e182b5 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:28.386021 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc59178-501e-4ef2-803a-5e051a221ea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:28.386915 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc59178-501e-4ef2-803a-5e051a221ea4 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:28.387206 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc59178-501e-4ef2-803a-5e051a221ea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:28.387484 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc59178-501e-4ef2-803a-5e051a221ea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:28.401142 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc59178-501e-4ef2-803a-5e051a221ea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:28.403232 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc59178-501e-4ef2-803a-5e051a221ea4 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:28.414551 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 993/2010] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:28.521167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a567b25-9518-4b40-a5ca-d4757fd67276 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:28.522716 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-1a567b25-9518-4b40-a5ca-d4757fd67276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:28.535964 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a567b25-9518-4b40-a5ca-d4757fd67276 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:28.541541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55301243-4651-42f2-a9fc-3a454caabf9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:28.542504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55301243-4651-42f2-a9fc-3a454caabf9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:28.542793 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55301243-4651-42f2-a9fc-3a454caabf9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:28.543073 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55301243-4651-42f2-a9fc-3a454caabf9f admin admin] PATH_INFO: `/v3/users/aaa08c8b3528410e931f6831bf6ed5d0` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:28.544264 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55301243-4651-42f2-a9fc-3a454caabf9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaa08c8b3528410e931f6831bf6ed5d0)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:28.546279 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55301243-4651-42f2-a9fc-3a454caabf9f admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:28.659534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc59178-501e-4ef2-803a-5e051a221ea4 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:28.661133 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-dfc59178-501e-4ef2-803a-5e051a221ea4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:28.675177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc59178-501e-4ef2-803a-5e051a221ea4 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:28.680303 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853222f-4140-49b4-b3f9-3af9836896ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:28.681666 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853222f-4140-49b4-b3f9-3af9836896ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:28.682048 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853222f-4140-49b4-b3f9-3af9836896ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:28.682529 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853222f-4140-49b4-b3f9-3af9836896ac admin admin] PATH_INFO: `/v3/projects/04eb80484f754d6fb5155d374929361f` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:28.683960 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853222f-4140-49b4-b3f9-3af9836896ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=04eb80484f754d6fb5155d374929361f)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:28.686031 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853222f-4140-49b4-b3f9-3af9836896ac admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:28.779283 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-55301243-4651-42f2-a9fc-3a454caabf9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaa08c8b3528410e931f6831bf6ed5d0'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:28.794717 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-55301243-4651-42f2-a9fc-3a454caabf9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaa08c8b3528410e931f6831bf6ed5d0'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:28.799781 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1853222f-4140-49b4-b3f9-3af9836896ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04eb80484f754d6fb5155d374929361f', 'request_id': 'req-1853222f-4140-49b4-b3f9-3af9836896ac'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:28.805841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1853222f-4140-49b4-b3f9-3af9836896ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04eb80484f754d6fb5155d374929361f', 'request_id': 'req-1853222f-4140-49b4-b3f9-3af9836896ac'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:28.884922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-55301243-4651-42f2-a9fc-3a454caabf9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaa08c8b3528410e931f6831bf6ed5d0'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:28.886040 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-55301243-4651-42f2-a9fc-3a454caabf9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaa08c8b3528410e931f6831bf6ed5d0'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:28.897205 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-55301243-4651-42f2-a9fc-3a454caabf9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaa08c8b3528410e931f6831bf6ed5d0'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:28.904517 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1018/2011] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:28 2022] DELETE /identity/v3/users/aaa08c8b3528410e931f6831bf6ed5d0 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:28.905493 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55301243-4651-42f2-a9fc-3a454caabf9f admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:28.906827 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-55301243-4651-42f2-a9fc-3a454caabf9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:28.935535 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:29.015293 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1853222f-4140-49b4-b3f9-3af9836896ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04eb80484f754d6fb5155d374929361f', 'request_id': 'req-1853222f-4140-49b4-b3f9-3af9836896ac'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:29.019642 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55301243-4651-42f2-a9fc-3a454caabf9f admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:29.028107 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a345521-c758-435a-8ae3-d5004213a7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:29.029045 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a345521-c758-435a-8ae3-d5004213a7d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:29.029502 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a345521-c758-435a-8ae3-d5004213a7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:29.029782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a345521-c758-435a-8ae3-d5004213a7d6 admin admin] PATH_INFO: `/v3/projects/3ae8fafc3b0842859e669dd38fda108b` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:29.031858 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a345521-c758-435a-8ae3-d5004213a7d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ae8fafc3b0842859e669dd38fda108b)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:29.033827 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a345521-c758-435a-8ae3-d5004213a7d6 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:29.043975 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1853222f-4140-49b4-b3f9-3af9836896ac admin admin] The token cache is being invalidate because project 04eb80484f754d6fb5155d374929361f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:51:29.044268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1853222f-4140-49b4-b3f9-3af9836896ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:29.044912 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1853222f-4140-49b4-b3f9-3af9836896ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:29.046756 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 994/2012] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:28 2022] DELETE /identity/v3/projects/04eb80484f754d6fb5155d374929361f => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:51:29.050036 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602818b7-5142-48b7-bb60-b51ba9d66308 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:29.050036 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602818b7-5142-48b7-bb60-b51ba9d66308 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:29.050036 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602818b7-5142-48b7-bb60-b51ba9d66308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:29.074575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-602818b7-5142-48b7-bb60-b51ba9d66308 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:29.075873 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-602818b7-5142-48b7-bb60-b51ba9d66308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:29.083756 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 995/2013] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:29.085235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2c7aa-fa25-40b3-a3bf-3d0308eeaa40 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:29.085752 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2c7aa-fa25-40b3-a3bf-3d0308eeaa40 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:29.086133 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c2c7aa-fa25-40b3-a3bf-3d0308eeaa40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:29.107433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-14c2c7aa-fa25-40b3-a3bf-3d0308eeaa40 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:29.108462 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-14c2c7aa-fa25-40b3-a3bf-3d0308eeaa40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:29.117079 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 996/2014] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:29.117882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2c7aa-fa25-40b3-a3bf-3d0308eeaa40 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:29.119150 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-14c2c7aa-fa25-40b3-a3bf-3d0308eeaa40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:29.142839 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-1a345521-c758-435a-8ae3-d5004213a7d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ae8fafc3b0842859e669dd38fda108b', 'request_id': 'req-1a345521-c758-435a-8ae3-d5004213a7d6'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:29.148410 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-1a345521-c758-435a-8ae3-d5004213a7d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ae8fafc3b0842859e669dd38fda108b', 'request_id': 'req-1a345521-c758-435a-8ae3-d5004213a7d6'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:29.155064 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-1a345521-c758-435a-8ae3-d5004213a7d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ae8fafc3b0842859e669dd38fda108b', 'request_id': 'req-1a345521-c758-435a-8ae3-d5004213a7d6'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:29.176860 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:29.193198 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c2c7aa-fa25-40b3-a3bf-3d0308eeaa40 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:29.232976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-1a345521-c758-435a-8ae3-d5004213a7d6 admin admin] The token cache is being invalidate because project 3ae8fafc3b0842859e669dd38fda108b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:51:29.233263 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-1a345521-c758-435a-8ae3-d5004213a7d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:29.233936 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-1a345521-c758-435a-8ae3-d5004213a7d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:29.235538 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1019/2015] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:28 2022] DELETE /identity/v3/projects/3ae8fafc3b0842859e669dd38fda108b => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:29.238560 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b41aa-0fab-48d3-8280-f3c1515f84ce None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:29.238560 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b41aa-0fab-48d3-8280-f3c1515f84ce None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:29.238560 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b41aa-0fab-48d3-8280-f3c1515f84ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:29.243480 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed3e41-3fe3-4fa0-9a5a-f7c96bf04e93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:29.244177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed3e41-3fe3-4fa0-9a5a-f7c96bf04e93 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:29.244466 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed3e41-3fe3-4fa0-9a5a-f7c96bf04e93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:29.244748 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed3e41-3fe3-4fa0-9a5a-f7c96bf04e93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:29.246829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-c8ed3e41-3fe3-4fa0-9a5a-f7c96bf04e93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:29.255819 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-786b41aa-0fab-48d3-8280-f3c1515f84ce None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:29.256919 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-786b41aa-0fab-48d3-8280-f3c1515f84ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:29.345506 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed3e41-3fe3-4fa0-9a5a-f7c96bf04e93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:29.347330 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed3e41-3fe3-4fa0-9a5a-f7c96bf04e93 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:29.353916 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1020/2016] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:29.359455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b41aa-0fab-48d3-8280-f3c1515f84ce None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:29.361417 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-786b41aa-0fab-48d3-8280-f3c1515f84ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:29.376459 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 997/2017] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:29.376459 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed3e41-3fe3-4fa0-9a5a-f7c96bf04e93 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:29.378841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786b41aa-0fab-48d3-8280-f3c1515f84ce None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:29.385421 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982eaae-6ba6-4f36-aaba-b0758580e02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:29.387049 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982eaae-6ba6-4f36-aaba-b0758580e02d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:29.387425 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982eaae-6ba6-4f36-aaba-b0758580e02d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:29.387863 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982eaae-6ba6-4f36-aaba-b0758580e02d admin admin] PATH_INFO: `/v3/users/a80533b0d89e49a3932ce6dec7dfe060` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:29.389412 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982eaae-6ba6-4f36-aaba-b0758580e02d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a80533b0d89e49a3932ce6dec7dfe060)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:29.391939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982eaae-6ba6-4f36-aaba-b0758580e02d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:29.395361 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed3e41-3fe3-4fa0-9a5a-f7c96bf04e93 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:29.409794 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c8076-934c-4a50-9590-87fe2acd4713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:29.411112 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c8076-934c-4a50-9590-87fe2acd4713 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:29.412402 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c8076-934c-4a50-9590-87fe2acd4713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:29.412402 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c8076-934c-4a50-9590-87fe2acd4713 admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:29.413423 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c8076-934c-4a50-9590-87fe2acd4713 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:51:29.413743 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c8076-934c-4a50-9590-87fe2acd4713 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:29.417180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c8076-934c-4a50-9590-87fe2acd4713 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:29.438533 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 998/2018] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:51:29.440588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c8076-934c-4a50-9590-87fe2acd4713 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:29.458867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c8076-934c-4a50-9590-87fe2acd4713 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:29.468631 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c2e38-616d-4b83-8d58-c74d177e7255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:29.468631 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c2e38-616d-4b83-8d58-c74d177e7255 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:29.468631 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c2e38-616d-4b83-8d58-c74d177e7255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:29.468631 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c2e38-616d-4b83-8d58-c74d177e7255 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:29.470348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54c2e38-616d-4b83-8d58-c74d177e7255 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:29.472508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54c2e38-616d-4b83-8d58-c74d177e7255 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:29.585655 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 999/2019] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:51:29 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:29.589575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c2e38-616d-4b83-8d58-c74d177e7255 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:29.659142 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c2e38-616d-4b83-8d58-c74d177e7255 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:29.664224 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5e8ab-b8e9-453e-ae11-2d3b2ec0a76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:29.665956 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5e8ab-b8e9-453e-ae11-2d3b2ec0a76e admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:29.666242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5e8ab-b8e9-453e-ae11-2d3b2ec0a76e admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:29.666664 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae5e8ab-b8e9-453e-ae11-2d3b2ec0a76e admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:29.667490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5e8ab-b8e9-453e-ae11-2d3b2ec0a76e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:29.669478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae5e8ab-b8e9-453e-ae11-2d3b2ec0a76e admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:29.672170 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-5982eaae-6ba6-4f36-aaba-b0758580e02d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a80533b0d89e49a3932ce6dec7dfe060'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:29.687447 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-5982eaae-6ba6-4f36-aaba-b0758580e02d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a80533b0d89e49a3932ce6dec7dfe060'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:29.769684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1000/2020] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:51:29 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:51:29.773695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5e8ab-b8e9-453e-ae11-2d3b2ec0a76e admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:29.784126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae5e8ab-b8e9-453e-ae11-2d3b2ec0a76e admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:29.788725 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b125b-8cb4-42f3-8b24-003bda34fa6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:29.789401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b125b-8cb4-42f3-8b24-003bda34fa6f admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:29.790368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b125b-8cb4-42f3-8b24-003bda34fa6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:29.790580 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b125b-8cb4-42f3-8b24-003bda34fa6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:29.793012 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-5982eaae-6ba6-4f36-aaba-b0758580e02d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a80533b0d89e49a3932ce6dec7dfe060'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:29.794902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-5982eaae-6ba6-4f36-aaba-b0758580e02d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a80533b0d89e49a3932ce6dec7dfe060'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:29.801203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b125b-8cb4-42f3-8b24-003bda34fa6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:29.802850 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b125b-8cb4-42f3-8b24-003bda34fa6f admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:29.804629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-5982eaae-6ba6-4f36-aaba-b0758580e02d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a80533b0d89e49a3932ce6dec7dfe060'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:29.811545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1021/2021] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:29 2022] DELETE /identity/v3/users/a80533b0d89e49a3932ce6dec7dfe060 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:29.816230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1001/2022] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:51:29.817505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982eaae-6ba6-4f36-aaba-b0758580e02d admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:29.818938 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-5982eaae-6ba6-4f36-aaba-b0758580e02d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:29.852968 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b125b-8cb4-42f3-8b24-003bda34fa6f admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:29.855487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-a92b125b-8cb4-42f3-8b24-003bda34fa6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:29.876266 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b125b-8cb4-42f3-8b24-003bda34fa6f admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:29.899833 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982eaae-6ba6-4f36-aaba-b0758580e02d admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:29.906100 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27da01-62c8-4a06-a342-a01f38ed5033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:29.906925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27da01-62c8-4a06-a342-a01f38ed5033 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:29.907303 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27da01-62c8-4a06-a342-a01f38ed5033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:29.907656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27da01-62c8-4a06-a342-a01f38ed5033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:29.908474 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27da01-62c8-4a06-a342-a01f38ed5033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:29.910295 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27da01-62c8-4a06-a342-a01f38ed5033 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:29.921873 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1022/2023] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:29.926291 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27da01-62c8-4a06-a342-a01f38ed5033 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:29.936892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27da01-62c8-4a06-a342-a01f38ed5033 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:29.946040 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b69476-2207-4ed1-96c4-d62b0797dc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:29.946969 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b69476-2207-4ed1-96c4-d62b0797dc17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:29.947254 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b69476-2207-4ed1-96c4-d62b0797dc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:29.947533 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b69476-2207-4ed1-96c4-d62b0797dc17 admin admin] PATH_INFO: `/v3/projects/05d4942fb11647f1b0dba5331a6399f7/users/0b73c2ab337040aba08c94a5da60849a/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:29.966046 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b69476-2207-4ed1-96c4-d62b0797dc17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05d4942fb11647f1b0dba5331a6399f7, user_id=0b73c2ab337040aba08c94a5da60849a, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:29.966949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faee52aa-0a96-4389-9d68-0f4e673ae0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:29.967833 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee52aa-0a96-4389-9d68-0f4e673ae0d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:29.968180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee52aa-0a96-4389-9d68-0f4e673ae0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:29.968684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee52aa-0a96-4389-9d68-0f4e673ae0d1 admin admin] PATH_INFO: `/v3/projects/081822f237724bdfab99001aebdb6452` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:29.969141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b69476-2207-4ed1-96c4-d62b0797dc17 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:29.969929 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee52aa-0a96-4389-9d68-0f4e673ae0d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=081822f237724bdfab99001aebdb6452)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:29.971870 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faee52aa-0a96-4389-9d68-0f4e673ae0d1 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:30.061573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1023/2024] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:29 2022] PUT /identity/v3/projects/05d4942fb11647f1b0dba5331a6399f7/users/0b73c2ab337040aba08c94a5da60849a/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:30.061573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b69476-2207-4ed1-96c4-d62b0797dc17 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:30.065667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-75b69476-2207-4ed1-96c4-d62b0797dc17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:30.133313 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-faee52aa-0a96-4389-9d68-0f4e673ae0d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '081822f237724bdfab99001aebdb6452', 'request_id': 'req-faee52aa-0a96-4389-9d68-0f4e673ae0d1'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:30.136975 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b69476-2207-4ed1-96c4-d62b0797dc17 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:30.139696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-faee52aa-0a96-4389-9d68-0f4e673ae0d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '081822f237724bdfab99001aebdb6452', 'request_id': 'req-faee52aa-0a96-4389-9d68-0f4e673ae0d1'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:30.209593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-faee52aa-0a96-4389-9d68-0f4e673ae0d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '081822f237724bdfab99001aebdb6452', 'request_id': 'req-faee52aa-0a96-4389-9d68-0f4e673ae0d1'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:30.212443 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b1a72-aa25-42f0-9e5c-f74afbc2ffab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:30.213482 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b1a72-aa25-42f0-9e5c-f74afbc2ffab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:30.213831 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b1a72-aa25-42f0-9e5c-f74afbc2ffab admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:30.214081 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b1a72-aa25-42f0-9e5c-f74afbc2ffab admin admin] PATH_INFO: `/v3/users/2801017cd40a45c2b9c82eec0c759717` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:30.215259 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6b1a72-aa25-42f0-9e5c-f74afbc2ffab admin admin] RBAC: Authorizing `identity:delete_user(user_id=2801017cd40a45c2b9c82eec0c759717)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:30.217223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6b1a72-aa25-42f0-9e5c-f74afbc2ffab admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:30.225523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-faee52aa-0a96-4389-9d68-0f4e673ae0d1 admin admin] The token cache is being invalidate because project 081822f237724bdfab99001aebdb6452 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:51:30.225821 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-faee52aa-0a96-4389-9d68-0f4e673ae0d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:30.226660 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-faee52aa-0a96-4389-9d68-0f4e673ae0d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:30.228790 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1002/2025] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:29 2022] DELETE /identity/v3/projects/081822f237724bdfab99001aebdb6452 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:30.230345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bef4bc-4080-445d-8de9-2a1af5d56c53 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:30.230609 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bef4bc-4080-445d-8de9-2a1af5d56c53 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:30.230925 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bef4bc-4080-445d-8de9-2a1af5d56c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:30.262228 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:30.263880 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-35bef4bc-4080-445d-8de9-2a1af5d56c53 None None] MFA Rules not processed for user `0b73c2ab337040aba08c94a5da60849a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:30.265220 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-35bef4bc-4080-445d-8de9-2a1af5d56c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:30.336470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1003/2026] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:30.338589 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d4e18-7016-47c8-8213-e1ca9087d8fa None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:30.338914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d4e18-7016-47c8-8213-e1ca9087d8fa None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:30.339197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546d4e18-7016-47c8-8213-e1ca9087d8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:30.360801 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-546d4e18-7016-47c8-8213-e1ca9087d8fa None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:30.369290 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-546d4e18-7016-47c8-8213-e1ca9087d8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:30.370568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1004/2027] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:30.371605 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d4e18-7016-47c8-8213-e1ca9087d8fa None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:30.372831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-546d4e18-7016-47c8-8213-e1ca9087d8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:30.385970 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546d4e18-7016-47c8-8213-e1ca9087d8fa None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:30.392910 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c792c00-12da-415c-9a39-9136547010dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:30.397460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c792c00-12da-415c-9a39-9136547010dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:30.397460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c792c00-12da-415c-9a39-9136547010dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:30.397460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c792c00-12da-415c-9a39-9136547010dd admin admin] PATH_INFO: `/v3/projects/3c802b45ef5f46af85aa28ddd56eee70` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:30.397460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c792c00-12da-415c-9a39-9136547010dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c802b45ef5f46af85aa28ddd56eee70)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:30.397460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c792c00-12da-415c-9a39-9136547010dd admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:30.544511 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-ae6b1a72-aa25-42f0-9e5c-f74afbc2ffab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2801017cd40a45c2b9c82eec0c759717'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:30.562395 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-ae6b1a72-aa25-42f0-9e5c-f74afbc2ffab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2801017cd40a45c2b9c82eec0c759717'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:30.575139 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-7c792c00-12da-415c-9a39-9136547010dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c802b45ef5f46af85aa28ddd56eee70', 'request_id': 'req-7c792c00-12da-415c-9a39-9136547010dd'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:30.580824 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-7c792c00-12da-415c-9a39-9136547010dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c802b45ef5f46af85aa28ddd56eee70', 'request_id': 'req-7c792c00-12da-415c-9a39-9136547010dd'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:30.659399 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-ae6b1a72-aa25-42f0-9e5c-f74afbc2ffab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2801017cd40a45c2b9c82eec0c759717'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:30.659943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-ae6b1a72-aa25-42f0-9e5c-f74afbc2ffab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2801017cd40a45c2b9c82eec0c759717'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:30.668112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-ae6b1a72-aa25-42f0-9e5c-f74afbc2ffab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2801017cd40a45c2b9c82eec0c759717'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:30.674214 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1024/2028] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:30 2022] DELETE /identity/v3/users/2801017cd40a45c2b9c82eec0c759717 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:30.675084 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b1a72-aa25-42f0-9e5c-f74afbc2ffab admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:30.676837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ae6b1a72-aa25-42f0-9e5c-f74afbc2ffab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:30.740503 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b1a72-aa25-42f0-9e5c-f74afbc2ffab admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:30.744175 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03aee2e-6a68-4c19-b727-a30a83001075 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:30.744758 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03aee2e-6a68-4c19-b727-a30a83001075 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:30.745092 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03aee2e-6a68-4c19-b727-a30a83001075 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:30.745378 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03aee2e-6a68-4c19-b727-a30a83001075 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:30.747161 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d03aee2e-6a68-4c19-b727-a30a83001075 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:30.751077 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-7c792c00-12da-415c-9a39-9136547010dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c802b45ef5f46af85aa28ddd56eee70', 'request_id': 'req-7c792c00-12da-415c-9a39-9136547010dd'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:30.766538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-7c792c00-12da-415c-9a39-9136547010dd admin admin] The token cache is being invalidate because project 3c802b45ef5f46af85aa28ddd56eee70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:51:30.767196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-7c792c00-12da-415c-9a39-9136547010dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:30.767873 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-7c792c00-12da-415c-9a39-9136547010dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:30.769909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1005/2029] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:30 2022] DELETE /identity/v3/projects/3c802b45ef5f46af85aa28ddd56eee70 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:30.770973 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6bced1-fca4-43bd-afce-e5bb4a810c68 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:30.771262 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6bced1-fca4-43bd-afce-e5bb4a810c68 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:30.771540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6bced1-fca4-43bd-afce-e5bb4a810c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:30.775246 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:30.795632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-ea6bced1-fca4-43bd-afce-e5bb4a810c68 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:30.797446 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-ea6bced1-fca4-43bd-afce-e5bb4a810c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:30.821845 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03aee2e-6a68-4c19-b727-a30a83001075 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:30.824000 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03aee2e-6a68-4c19-b727-a30a83001075 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:30.835611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1025/2030] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:30.838166 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd12c8-827a-4f5f-9920-62f6dbd914d9 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:30.838291 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd12c8-827a-4f5f-9920-62f6dbd914d9 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:30.838487 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd12c8-827a-4f5f-9920-62f6dbd914d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:30.857867 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-efcd12c8-827a-4f5f-9920-62f6dbd914d9 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:30.859199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-efcd12c8-827a-4f5f-9920-62f6dbd914d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:30.874679 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1006/2031] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:30.876403 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89504c-09d7-4010-a671-4293d7b7bcfc None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:30.876734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89504c-09d7-4010-a671-4293d7b7bcfc None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:30.877047 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89504c-09d7-4010-a671-4293d7b7bcfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:30.900848 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-bc89504c-09d7-4010-a671-4293d7b7bcfc None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:30.903644 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-bc89504c-09d7-4010-a671-4293d7b7bcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:30.913657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1007/2032] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:30.914611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89504c-09d7-4010-a671-4293d7b7bcfc None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:30.927777 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89504c-09d7-4010-a671-4293d7b7bcfc None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:30.933840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268de1f9-5bcd-4804-a20d-9034a3432ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:30.934530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268de1f9-5bcd-4804-a20d-9034a3432ee0 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:30.934818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268de1f9-5bcd-4804-a20d-9034a3432ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:30.935098 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268de1f9-5bcd-4804-a20d-9034a3432ee0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:30.935908 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268de1f9-5bcd-4804-a20d-9034a3432ee0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:51:30.936359 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268de1f9-5bcd-4804-a20d-9034a3432ee0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:30.938592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268de1f9-5bcd-4804-a20d-9034a3432ee0 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:30.943123 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1026/2033] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:30.943123 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ad374-7307-4094-a54a-88996ba11607 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:30.943123 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ad374-7307-4094-a54a-88996ba11607 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:30.943123 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ad374-7307-4094-a54a-88996ba11607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:30.962949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1008/2034] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:51:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:51:30.963849 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268de1f9-5bcd-4804-a20d-9034a3432ee0 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:30.966567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-268de1f9-5bcd-4804-a20d-9034a3432ee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:30.971044 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-438ad374-7307-4094-a54a-88996ba11607 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:30.971938 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-438ad374-7307-4094-a54a-88996ba11607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:30.981319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1027/2035] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:30.982137 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ad374-7307-4094-a54a-88996ba11607 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:30.998001 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ad374-7307-4094-a54a-88996ba11607 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:31.001233 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db2d93-8a66-4c3e-85fd-b794649a7b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:31.002016 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db2d93-8a66-4c3e-85fd-b794649a7b69 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:31.003320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db2d93-8a66-4c3e-85fd-b794649a7b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:31.003730 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db2d93-8a66-4c3e-85fd-b794649a7b69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:31.004640 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db2d93-8a66-4c3e-85fd-b794649a7b69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:31.006723 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db2d93-8a66-4c3e-85fd-b794649a7b69 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:31.046481 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268de1f9-5bcd-4804-a20d-9034a3432ee0 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:31.050189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd19f1d-e82c-4ccb-8ac5-3e523cae573e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:31.050856 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd19f1d-e82c-4ccb-8ac5-3e523cae573e service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:31.051139 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd19f1d-e82c-4ccb-8ac5-3e523cae573e service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:31.051422 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd19f1d-e82c-4ccb-8ac5-3e523cae573e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:31.062133 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd19f1d-e82c-4ccb-8ac5-3e523cae573e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:31.063698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd19f1d-e82c-4ccb-8ac5-3e523cae573e service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:31.073983 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1009/2036] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:31.074807 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd19f1d-e82c-4ccb-8ac5-3e523cae573e service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:31.084121 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd19f1d-e82c-4ccb-8ac5-3e523cae573e service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:31.089374 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01850e00-4f8b-42ca-85b8-4352be7a98a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:31.090463 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01850e00-4f8b-42ca-85b8-4352be7a98a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:31.091065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01850e00-4f8b-42ca-85b8-4352be7a98a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:31.091403 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01850e00-4f8b-42ca-85b8-4352be7a98a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:31.092186 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01850e00-4f8b-42ca-85b8-4352be7a98a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:51:31.092491 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01850e00-4f8b-42ca-85b8-4352be7a98a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:31.094418 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01850e00-4f8b-42ca-85b8-4352be7a98a0 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:31.108990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1010/2037] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:51:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:51:31.111375 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01850e00-4f8b-42ca-85b8-4352be7a98a0 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:31.125766 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01850e00-4f8b-42ca-85b8-4352be7a98a0 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:31.131932 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1028/2038] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:51:30 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:31.132090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db2d93-8a66-4c3e-85fd-b794649a7b69 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:31.136287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-10db2d93-8a66-4c3e-85fd-b794649a7b69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:31.205615 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40fdc5-044a-4135-aa5b-56d1239a26fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:31.206280 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40fdc5-044a-4135-aa5b-56d1239a26fe admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:31.206672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40fdc5-044a-4135-aa5b-56d1239a26fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:31.206847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa40fdc5-044a-4135-aa5b-56d1239a26fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:31.207548 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40fdc5-044a-4135-aa5b-56d1239a26fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:31.209491 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa40fdc5-044a-4135-aa5b-56d1239a26fe admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:31.223677 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db2d93-8a66-4c3e-85fd-b794649a7b69 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:31.229197 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d92898-1885-4863-9adf-f82d4d7bb060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:31.230066 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d92898-1885-4863-9adf-f82d4d7bb060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:31.230346 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d92898-1885-4863-9adf-f82d4d7bb060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:31.230618 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d92898-1885-4863-9adf-f82d4d7bb060 admin admin] PATH_INFO: `/v3/users/10369664f1914c51893eef1929b7d26b` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:31.231837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d92898-1885-4863-9adf-f82d4d7bb060 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10369664f1914c51893eef1929b7d26b)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:31.233818 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d92898-1885-4863-9adf-f82d4d7bb060 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:31.341205 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1011/2039] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:51:31 2022] POST /identity/v3/projects => generated 355 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:51:31.341205 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40fdc5-044a-4135-aa5b-56d1239a26fe admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:31.395000 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:31.418734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa40fdc5-044a-4135-aa5b-56d1239a26fe admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:31.424028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4f54a-03e8-4163-9303-7029136bc2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:31.424724 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4f54a-03e8-4163-9303-7029136bc2e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:31.425038 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4f54a-03e8-4163-9303-7029136bc2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:31.429449 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c4f54a-03e8-4163-9303-7029136bc2e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:31.429449 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4f54a-03e8-4163-9303-7029136bc2e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:31.429449 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c4f54a-03e8-4163-9303-7029136bc2e0 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:31.515069 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-23d92898-1885-4863-9adf-f82d4d7bb060 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10369664f1914c51893eef1929b7d26b'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:31.527951 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-23d92898-1885-4863-9adf-f82d4d7bb060 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10369664f1914c51893eef1929b7d26b'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:31.531165 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:31.550509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1012/2040] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:51:31 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:31.554181 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4f54a-03e8-4163-9303-7029136bc2e0 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:31.582160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c4f54a-03e8-4163-9303-7029136bc2e0 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:31.587961 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0603a2-0415-42ab-9b3c-468364d4b175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:31.588683 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0603a2-0415-42ab-9b3c-468364d4b175 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:31.588998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0603a2-0415-42ab-9b3c-468364d4b175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:31.589289 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0603a2-0415-42ab-9b3c-468364d4b175 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:31.590082 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0603a2-0415-42ab-9b3c-468364d4b175 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:31.592113 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0603a2-0415-42ab-9b3c-468364d4b175 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:31.621360 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-23d92898-1885-4863-9adf-f82d4d7bb060 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10369664f1914c51893eef1929b7d26b'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:31.622060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-23d92898-1885-4863-9adf-f82d4d7bb060 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10369664f1914c51893eef1929b7d26b'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:31.630061 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-23d92898-1885-4863-9adf-f82d4d7bb060 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10369664f1914c51893eef1929b7d26b'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:31.639965 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1029/2041] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:31 2022] DELETE /identity/v3/users/10369664f1914c51893eef1929b7d26b => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:51:31.639965 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d92898-1885-4863-9adf-f82d4d7bb060 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:31.643966 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-23d92898-1885-4863-9adf-f82d4d7bb060 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:31.720042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d92898-1885-4863-9adf-f82d4d7bb060 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:31.720557 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1013/2042] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:51:31 2022] POST /identity/v3/users => generated 278 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:31.721439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0603a2-0415-42ab-9b3c-468364d4b175 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:31.723953 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-8a0603a2-0415-42ab-9b3c-468364d4b175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:31.726257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5855a-793f-4cfe-a617-bfafe1ea56e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:31.727841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5855a-793f-4cfe-a617-bfafe1ea56e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:31.728128 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5855a-793f-4cfe-a617-bfafe1ea56e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:31.728618 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5855a-793f-4cfe-a617-bfafe1ea56e7 admin admin] PATH_INFO: `/v3/projects/e614399115ec4911a8557fde2d4b55f2` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:31.730292 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5855a-793f-4cfe-a617-bfafe1ea56e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e614399115ec4911a8557fde2d4b55f2)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:31.732259 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5855a-793f-4cfe-a617-bfafe1ea56e7 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:31.737590 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0603a2-0415-42ab-9b3c-468364d4b175 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:31.744086 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698298d0-8f40-4580-8266-e71a346ce45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:31.749478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698298d0-8f40-4580-8266-e71a346ce45b admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:31.749478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698298d0-8f40-4580-8266-e71a346ce45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:31.749478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698298d0-8f40-4580-8266-e71a346ce45b admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:31.749478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698298d0-8f40-4580-8266-e71a346ce45b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:31.749478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698298d0-8f40-4580-8266-e71a346ce45b admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:31.763922 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1014/2043] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:31.764853 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698298d0-8f40-4580-8266-e71a346ce45b admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:31.767678 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-698298d0-8f40-4580-8266-e71a346ce45b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:31.780691 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698298d0-8f40-4580-8266-e71a346ce45b admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:31.786796 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda3bab-d9cc-4939-8cfa-0246986a3703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:31.787567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda3bab-d9cc-4939-8cfa-0246986a3703 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:31.787929 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda3bab-d9cc-4939-8cfa-0246986a3703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:31.788279 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda3bab-d9cc-4939-8cfa-0246986a3703 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:31.789173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cda3bab-d9cc-4939-8cfa-0246986a3703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:31.791326 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cda3bab-d9cc-4939-8cfa-0246986a3703 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:31.801700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1015/2044] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:31.802525 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda3bab-d9cc-4939-8cfa-0246986a3703 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:31.812434 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda3bab-d9cc-4939-8cfa-0246986a3703 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:31.817058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e4f0f-f6d3-4398-91a8-b353450aa2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:31.817781 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e4f0f-f6d3-4398-91a8-b353450aa2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:31.818068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e4f0f-f6d3-4398-91a8-b353450aa2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:31.818350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e4f0f-f6d3-4398-91a8-b353450aa2f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:31.819072 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e4f0f-f6d3-4398-91a8-b353450aa2f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:31.820661 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e4f0f-f6d3-4398-91a8-b353450aa2f1 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:31.831473 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1016/2045] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:31.835030 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e4f0f-f6d3-4398-91a8-b353450aa2f1 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:31.842326 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e4f0f-f6d3-4398-91a8-b353450aa2f1 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:31.847243 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be864634-bd23-4885-94c7-8f7bfe2a959c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:31.847881 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be864634-bd23-4885-94c7-8f7bfe2a959c admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:31.848169 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be864634-bd23-4885-94c7-8f7bfe2a959c admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:31.848446 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be864634-bd23-4885-94c7-8f7bfe2a959c admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:31.849189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be864634-bd23-4885-94c7-8f7bfe2a959c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:31.850761 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be864634-bd23-4885-94c7-8f7bfe2a959c admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:31.864022 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1017/2046] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:31.864022 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be864634-bd23-4885-94c7-8f7bfe2a959c admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:31.875324 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c2e5855a-793f-4cfe-a617-bfafe1ea56e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e614399115ec4911a8557fde2d4b55f2', 'request_id': 'req-c2e5855a-793f-4cfe-a617-bfafe1ea56e7'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:31.875541 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be864634-bd23-4885-94c7-8f7bfe2a959c admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:31.881941 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c2e5855a-793f-4cfe-a617-bfafe1ea56e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e614399115ec4911a8557fde2d4b55f2', 'request_id': 'req-c2e5855a-793f-4cfe-a617-bfafe1ea56e7'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:31.890596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c2e5855a-793f-4cfe-a617-bfafe1ea56e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e614399115ec4911a8557fde2d4b55f2', 'request_id': 'req-c2e5855a-793f-4cfe-a617-bfafe1ea56e7'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:31.939344 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc5038-bd39-46fb-9ab5-d959261a7cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:31.940247 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc5038-bd39-46fb-9ab5-d959261a7cc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:31.940534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc5038-bd39-46fb-9ab5-d959261a7cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:31.940815 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc5038-bd39-46fb-9ab5-d959261a7cc6 admin admin] PATH_INFO: `/v3/projects/0ca4fe16a0204d0db4289e293771581b/users/c0f12003099f4f80bb89c2a335d231ca/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:31.955508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc5038-bd39-46fb-9ab5-d959261a7cc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ca4fe16a0204d0db4289e293771581b, user_id=c0f12003099f4f80bb89c2a335d231ca, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:31.958391 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc5038-bd39-46fb-9ab5-d959261a7cc6 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:31.975809 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c2e5855a-793f-4cfe-a617-bfafe1ea56e7 admin admin] The token cache is being invalidate because project e614399115ec4911a8557fde2d4b55f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:51:31.976233 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c2e5855a-793f-4cfe-a617-bfafe1ea56e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:31.976902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c2e5855a-793f-4cfe-a617-bfafe1ea56e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:31.978651 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1030/2047] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:31 2022] DELETE /identity/v3/projects/e614399115ec4911a8557fde2d4b55f2 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:31.979454 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5855a-793f-4cfe-a617-bfafe1ea56e7 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:31.980680 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-c2e5855a-793f-4cfe-a617-bfafe1ea56e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:31.994472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5855a-793f-4cfe-a617-bfafe1ea56e7 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:31.999302 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e1032d-4b5a-4c02-a405-93c78d268aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:32.000226 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1032d-4b5a-4c02-a405-93c78d268aa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:32.000533 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1032d-4b5a-4c02-a405-93c78d268aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:32.001432 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1032d-4b5a-4c02-a405-93c78d268aa0 admin admin] PATH_INFO: `/v3/projects/51efa3774bfa4adaafbf26a0bb090690/users/2b58d2c0879a46af818c44d4f34a1ece/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:32.016660 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e1032d-4b5a-4c02-a405-93c78d268aa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51efa3774bfa4adaafbf26a0bb090690, user_id=2b58d2c0879a46af818c44d4f34a1ece, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:32.019376 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e1032d-4b5a-4c02-a405-93c78d268aa0 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:32.055925 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1018/2048] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:31 2022] PUT /identity/v3/projects/0ca4fe16a0204d0db4289e293771581b/users/c0f12003099f4f80bb89c2a335d231ca/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:32.056780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc5038-bd39-46fb-9ab5-d959261a7cc6 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:32.058399 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-4dcc5038-bd39-46fb-9ab5-d959261a7cc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:32.116441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1031/2049] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:31 2022] PUT /identity/v3/projects/51efa3774bfa4adaafbf26a0bb090690/users/2b58d2c0879a46af818c44d4f34a1ece/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:32.118657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e1032d-4b5a-4c02-a405-93c78d268aa0 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:32.126599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc5038-bd39-46fb-9ab5-d959261a7cc6 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:32.127086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-23e1032d-4b5a-4c02-a405-93c78d268aa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:32.184018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c71ba7-3ae1-489d-90d5-92a4742586de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:32.185255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c71ba7-3ae1-489d-90d5-92a4742586de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:32.185544 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c71ba7-3ae1-489d-90d5-92a4742586de admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:32.185824 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c71ba7-3ae1-489d-90d5-92a4742586de admin admin] PATH_INFO: `/v3/projects/c4362059e243479193d5a02e2e5611a1` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:32.187011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c71ba7-3ae1-489d-90d5-92a4742586de admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4362059e243479193d5a02e2e5611a1)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:32.188967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c71ba7-3ae1-489d-90d5-92a4742586de admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:32.193007 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e1032d-4b5a-4c02-a405-93c78d268aa0 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:32.199353 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e8fc3f-b04d-4906-a96f-b25211a42ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:32.200275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8fc3f-b04d-4906-a96f-b25211a42ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:32.202011 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8fc3f-b04d-4906-a96f-b25211a42ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:32.202011 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e8fc3f-b04d-4906-a96f-b25211a42ea4 admin admin] PATH_INFO: `/v3/users/e74d0fc799e2402686e4360952a4930a` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:32.202203 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8fc3f-b04d-4906-a96f-b25211a42ea4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e74d0fc799e2402686e4360952a4930a)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:32.204174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e8fc3f-b04d-4906-a96f-b25211a42ea4 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:32.318262 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-66c71ba7-3ae1-489d-90d5-92a4742586de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4362059e243479193d5a02e2e5611a1', 'request_id': 'req-66c71ba7-3ae1-489d-90d5-92a4742586de'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:32.323720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-66c71ba7-3ae1-489d-90d5-92a4742586de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4362059e243479193d5a02e2e5611a1', 'request_id': 'req-66c71ba7-3ae1-489d-90d5-92a4742586de'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:32.425702 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-66c71ba7-3ae1-489d-90d5-92a4742586de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4362059e243479193d5a02e2e5611a1', 'request_id': 'req-66c71ba7-3ae1-489d-90d5-92a4742586de'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:32.439088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-66c71ba7-3ae1-489d-90d5-92a4742586de admin admin] The token cache is being invalidate because project c4362059e243479193d5a02e2e5611a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:51:32.439390 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-66c71ba7-3ae1-489d-90d5-92a4742586de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:32.440736 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-66c71ba7-3ae1-489d-90d5-92a4742586de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:32.440879 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-93e8fc3f-b04d-4906-a96f-b25211a42ea4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e74d0fc799e2402686e4360952a4930a'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:32.441905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1019/2050] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:32 2022] DELETE /identity/v3/projects/c4362059e243479193d5a02e2e5611a1 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:32.446537 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466e380-99b4-47ec-97cc-d01d7b84df34 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:32.446849 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466e380-99b4-47ec-97cc-d01d7b84df34 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:32.447140 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466e380-99b4-47ec-97cc-d01d7b84df34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:32.453466 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-93e8fc3f-b04d-4906-a96f-b25211a42ea4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e74d0fc799e2402686e4360952a4930a'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:32.465511 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-8466e380-99b4-47ec-97cc-d01d7b84df34 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:32.466717 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-8466e380-99b4-47ec-97cc-d01d7b84df34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:32.516886 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-93e8fc3f-b04d-4906-a96f-b25211a42ea4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e74d0fc799e2402686e4360952a4930a'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:32.517726 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-93e8fc3f-b04d-4906-a96f-b25211a42ea4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e74d0fc799e2402686e4360952a4930a'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:32.526286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-93e8fc3f-b04d-4906-a96f-b25211a42ea4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e74d0fc799e2402686e4360952a4930a'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:32.529615 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1020/2051] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:32.530932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450341e-31fa-48b5-9af1-96b59ec398f6 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:32.531296 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450341e-31fa-48b5-9af1-96b59ec398f6 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:32.531498 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450341e-31fa-48b5-9af1-96b59ec398f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:32.535440 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1032/2052] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:32 2022] DELETE /identity/v3/users/e74d0fc799e2402686e4360952a4930a => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:32.536816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53578809-ec9c-4f39-9fb7-a351363a5c76 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:32.537139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53578809-ec9c-4f39-9fb7-a351363a5c76 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:32.537446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53578809-ec9c-4f39-9fb7-a351363a5c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:32.558546 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-53578809-ec9c-4f39-9fb7-a351363a5c76 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:32.559795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-53578809-ec9c-4f39-9fb7-a351363a5c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:32.560819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-e450341e-31fa-48b5-9af1-96b59ec398f6 None None] MFA Rules not processed for user `2b58d2c0879a46af818c44d4f34a1ece`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:32.563175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-e450341e-31fa-48b5-9af1-96b59ec398f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:32.578672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:32.636387 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1033/2053] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:32.636387 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53578809-ec9c-4f39-9fb7-a351363a5c76 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:32.636387 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-53578809-ec9c-4f39-9fb7-a351363a5c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:32.636700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1021/2054] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:32 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:32.638620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4d24c-5557-4c31-a8e5-05d3a36d9fe2 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:32.639065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4d24c-5557-4c31-a8e5-05d3a36d9fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:32.639356 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4d24c-5557-4c31-a8e5-05d3a36d9fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:32.656689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:32.665618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-d5f4d24c-5557-4c31-a8e5-05d3a36d9fe2 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:32.666781 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d5f4d24c-5557-4c31-a8e5-05d3a36d9fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:32.678954 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1022/2055] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:32.678954 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4d24c-5557-4c31-a8e5-05d3a36d9fe2 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:32.682589 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d5f4d24c-5557-4c31-a8e5-05d3a36d9fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:32.694544 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4d24c-5557-4c31-a8e5-05d3a36d9fe2 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:32.700507 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d02620-54fa-4882-a7da-667bfd4eb77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:32.701292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d02620-54fa-4882-a7da-667bfd4eb77f admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:32.701653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d02620-54fa-4882-a7da-667bfd4eb77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:32.702003 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53578809-ec9c-4f39-9fb7-a351363a5c76 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:32.702112 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d02620-54fa-4882-a7da-667bfd4eb77f admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:32.702829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d02620-54fa-4882-a7da-667bfd4eb77f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:32.705075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d02620-54fa-4882-a7da-667bfd4eb77f admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:32.705835 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39993a-6394-47ab-abad-af91fa298b3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:32.706460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39993a-6394-47ab-abad-af91fa298b3d service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:32.706744 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39993a-6394-47ab-abad-af91fa298b3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:32.707024 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39993a-6394-47ab-abad-af91fa298b3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:32.717857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39993a-6394-47ab-abad-af91fa298b3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:32.719579 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39993a-6394-47ab-abad-af91fa298b3d service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:32.730099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1034/2056] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:32.731003 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39993a-6394-47ab-abad-af91fa298b3d service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:32.740524 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39993a-6394-47ab-abad-af91fa298b3d service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:32.746863 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168333b1-70cf-458f-a402-93459e6f44b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:32.747653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168333b1-70cf-458f-a402-93459e6f44b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:32.748081 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168333b1-70cf-458f-a402-93459e6f44b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:32.751644 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168333b1-70cf-458f-a402-93459e6f44b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:32.751644 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168333b1-70cf-458f-a402-93459e6f44b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:51:32.751644 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168333b1-70cf-458f-a402-93459e6f44b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:32.751644 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168333b1-70cf-458f-a402-93459e6f44b3 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:32.766692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1035/2057] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:51:32.767699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168333b1-70cf-458f-a402-93459e6f44b3 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:32.777685 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168333b1-70cf-458f-a402-93459e6f44b3 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:32.782423 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff81ec2-924c-454f-8eff-da1b877e31b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:32.783068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff81ec2-924c-454f-8eff-da1b877e31b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:32.783377 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff81ec2-924c-454f-8eff-da1b877e31b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:32.783651 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff81ec2-924c-454f-8eff-da1b877e31b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:32.784507 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff81ec2-924c-454f-8eff-da1b877e31b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:32.786531 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff81ec2-924c-454f-8eff-da1b877e31b9 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:32.848743 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1023/2058] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:51:32 2022] POST /identity/v3/projects => generated 363 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:32.849891 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d02620-54fa-4882-a7da-667bfd4eb77f admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:32.882968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1036/2059] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:51:32 2022] POST /identity/v3/projects => generated 391 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:32.888493 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff81ec2-924c-454f-8eff-da1b877e31b9 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:32.923173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d02620-54fa-4882-a7da-667bfd4eb77f admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:32.984781 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff81ec2-924c-454f-8eff-da1b877e31b9 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:32.990978 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb7dcf-b58b-45e7-8288-6bb6d6099b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:32.991635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb7dcf-b58b-45e7-8288-6bb6d6099b37 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:32.992304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb7dcf-b58b-45e7-8288-6bb6d6099b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:32.992428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b3366-1909-42a2-8d94-4a7bb41a6bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:32.993173 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb7dcf-b58b-45e7-8288-6bb6d6099b37 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:32.993611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb7dcf-b58b-45e7-8288-6bb6d6099b37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:32.993953 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b3366-1909-42a2-8d94-4a7bb41a6bc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:32.994312 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b3366-1909-42a2-8d94-4a7bb41a6bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:32.994664 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28b3366-1909-42a2-8d94-4a7bb41a6bc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:32.995433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b3366-1909-42a2-8d94-4a7bb41a6bc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:32.995954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb7dcf-b58b-45e7-8288-6bb6d6099b37 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:32.997503 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28b3366-1909-42a2-8d94-4a7bb41a6bc5 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:33.100355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1037/2060] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:51:32 2022] POST /identity/v3/users => generated 296 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:33.102521 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb7dcf-b58b-45e7-8288-6bb6d6099b37 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:33.104977 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1024/2061] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:51:32 2022] POST /identity/v3/users => generated 288 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:51:33.105616 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-e5cb7dcf-b58b-45e7-8288-6bb6d6099b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:33.106028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b3366-1909-42a2-8d94-4a7bb41a6bc5 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:33.116795 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28b3366-1909-42a2-8d94-4a7bb41a6bc5 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:33.124788 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb7dcf-b58b-45e7-8288-6bb6d6099b37 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:33.127324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d69a3-3706-422b-b774-d91b20995e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:33.128047 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d69a3-3706-422b-b774-d91b20995e75 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:33.135443 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d69a3-3706-422b-b774-d91b20995e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:33.135443 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d69a3-3706-422b-b774-d91b20995e75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:33.135443 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d69a3-3706-422b-b774-d91b20995e75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:33.135925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a105e-a6ec-4a33-9beb-eeea91b646b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:33.136019 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d69a3-3706-422b-b774-d91b20995e75 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:33.136092 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a105e-a6ec-4a33-9beb-eeea91b646b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:33.136092 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a105e-a6ec-4a33-9beb-eeea91b646b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:33.136092 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a105e-a6ec-4a33-9beb-eeea91b646b8 admin admin] PATH_INFO: `/v3/projects/76ba3673912842d3b9ab1fdaba193e9d` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:33.136092 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a105e-a6ec-4a33-9beb-eeea91b646b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76ba3673912842d3b9ab1fdaba193e9d)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:33.136915 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a105e-a6ec-4a33-9beb-eeea91b646b8 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:33.146485 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1025/2062] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:33.146485 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d69a3-3706-422b-b774-d91b20995e75 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:33.160997 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d69a3-3706-422b-b774-d91b20995e75 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:33.166832 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1e592-3468-47f8-ad18-05af7519bd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:33.167658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1e592-3468-47f8-ad18-05af7519bd19 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:33.168246 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1e592-3468-47f8-ad18-05af7519bd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:33.168624 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb1e592-3468-47f8-ad18-05af7519bd19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:33.169522 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1e592-3468-47f8-ad18-05af7519bd19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:33.171332 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb1e592-3468-47f8-ad18-05af7519bd19 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:33.182964 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1026/2063] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:33.183952 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1e592-3468-47f8-ad18-05af7519bd19 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:33.193877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb1e592-3468-47f8-ad18-05af7519bd19 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:33.202834 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbf00f-662a-46c5-acd0-ec53667ec64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:33.203732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbf00f-662a-46c5-acd0-ec53667ec64f admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:33.204013 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbf00f-662a-46c5-acd0-ec53667ec64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:33.204428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbf00f-662a-46c5-acd0-ec53667ec64f admin admin] PATH_INFO: `/v3/projects/30f12cc1e28c4da295dbe1b8168e3b27/users/acf77182176441ebb30e1e8c13c7f0da/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:33.215509 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:33.229552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cbf00f-662a-46c5-acd0-ec53667ec64f admin admin] RBAC: Authorizing `identity:create_grant(project_id=30f12cc1e28c4da295dbe1b8168e3b27, user_id=acf77182176441ebb30e1e8c13c7f0da, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:33.231766 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cbf00f-662a-46c5-acd0-ec53667ec64f admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:33.338730 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1027/2064] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:33 2022] PUT /identity/v3/projects/30f12cc1e28c4da295dbe1b8168e3b27/users/acf77182176441ebb30e1e8c13c7f0da/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:33.344878 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbf00f-662a-46c5-acd0-ec53667ec64f admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:33.350144 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:33.351873 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-2e8a105e-a6ec-4a33-9beb-eeea91b646b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76ba3673912842d3b9ab1fdaba193e9d', 'request_id': 'req-2e8a105e-a6ec-4a33-9beb-eeea91b646b8'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:33.361380 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-2e8a105e-a6ec-4a33-9beb-eeea91b646b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76ba3673912842d3b9ab1fdaba193e9d', 'request_id': 'req-2e8a105e-a6ec-4a33-9beb-eeea91b646b8'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:33.371338 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-2e8a105e-a6ec-4a33-9beb-eeea91b646b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76ba3673912842d3b9ab1fdaba193e9d', 'request_id': 'req-2e8a105e-a6ec-4a33-9beb-eeea91b646b8'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:33.420344 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbf00f-662a-46c5-acd0-ec53667ec64f admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:33.426703 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a8b0d-e6fe-40a8-bc57-046fc3b4e239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:33.427401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a8b0d-e6fe-40a8-bc57-046fc3b4e239 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:33.427668 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a8b0d-e6fe-40a8-bc57-046fc3b4e239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:33.427943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a8b0d-e6fe-40a8-bc57-046fc3b4e239 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:33.428672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a8b0d-e6fe-40a8-bc57-046fc3b4e239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:33.430903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192a8b0d-e6fe-40a8-bc57-046fc3b4e239 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:33.441864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1028/2065] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:33.442811 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a8b0d-e6fe-40a8-bc57-046fc3b4e239 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:33.452971 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a8b0d-e6fe-40a8-bc57-046fc3b4e239 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:33.457559 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0cf71a-cc94-49e5-8dc0-0298504a09d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:33.458195 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0cf71a-cc94-49e5-8dc0-0298504a09d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:33.458477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0cf71a-cc94-49e5-8dc0-0298504a09d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:33.458753 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0cf71a-cc94-49e5-8dc0-0298504a09d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:33.459466 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0cf71a-cc94-49e5-8dc0-0298504a09d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:33.461099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0cf71a-cc94-49e5-8dc0-0298504a09d2 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:33.471219 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1029/2066] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:33.471921 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0cf71a-cc94-49e5-8dc0-0298504a09d2 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:33.481229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0cf71a-cc94-49e5-8dc0-0298504a09d2 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:33.483535 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-2e8a105e-a6ec-4a33-9beb-eeea91b646b8 admin admin] The token cache is being invalidate because project 76ba3673912842d3b9ab1fdaba193e9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:51:33.483830 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-2e8a105e-a6ec-4a33-9beb-eeea91b646b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:33.484497 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-2e8a105e-a6ec-4a33-9beb-eeea91b646b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:33.486009 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331073b4-e250-401b-bc43-f3dc29772668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:33.486137 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1038/2067] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:33 2022] DELETE /identity/v3/projects/76ba3673912842d3b9ab1fdaba193e9d => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:33.486843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a105e-a6ec-4a33-9beb-eeea91b646b8 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:33.486945 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331073b4-e250-401b-bc43-f3dc29772668 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:33.487139 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331073b4-e250-401b-bc43-f3dc29772668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:33.487431 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331073b4-e250-401b-bc43-f3dc29772668 admin admin] PATH_INFO: `/v3/projects/3388fd386824477385e3ad6baa4cae56/users/a13d2c50c5204c859872e99bc45c6566/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:33.490421 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-2e8a105e-a6ec-4a33-9beb-eeea91b646b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:33.503292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331073b4-e250-401b-bc43-f3dc29772668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3388fd386824477385e3ad6baa4cae56, user_id=a13d2c50c5204c859872e99bc45c6566, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:33.507094 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331073b4-e250-401b-bc43-f3dc29772668 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:33.588215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1030/2068] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:33 2022] PUT /identity/v3/projects/3388fd386824477385e3ad6baa4cae56/users/a13d2c50c5204c859872e99bc45c6566/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:33.588659 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a105e-a6ec-4a33-9beb-eeea91b646b8 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:33.589647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498e2d9-1b32-4845-a3b7-51020389f034 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:33.589876 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498e2d9-1b32-4845-a3b7-51020389f034 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:33.590155 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6498e2d9-1b32-4845-a3b7-51020389f034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:33.645740 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-6498e2d9-1b32-4845-a3b7-51020389f034 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:33.646978 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-6498e2d9-1b32-4845-a3b7-51020389f034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:33.660288 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d53d4d-7352-4bf3-b9af-445e83cfa6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:33.661659 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d53d4d-7352-4bf3-b9af-445e83cfa6cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:33.662147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d53d4d-7352-4bf3-b9af-445e83cfa6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:33.662622 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d53d4d-7352-4bf3-b9af-445e83cfa6cc admin admin] PATH_INFO: `/v3/projects/30f12cc1e28c4da295dbe1b8168e3b27/users/acf77182176441ebb30e1e8c13c7f0da/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:33.665243 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d53d4d-7352-4bf3-b9af-445e83cfa6cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=30f12cc1e28c4da295dbe1b8168e3b27, user_id=acf77182176441ebb30e1e8c13c7f0da, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:33.668456 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d53d4d-7352-4bf3-b9af-445e83cfa6cc admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:33.681551 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1039/2069] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:33 2022] PUT /identity/v3/projects/30f12cc1e28c4da295dbe1b8168e3b27/users/acf77182176441ebb30e1e8c13c7f0da/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:33.682907 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1775f7-7fdb-4732-a86f-82de9e89b6eb None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:33.683201 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1775f7-7fdb-4732-a86f-82de9e89b6eb None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:33.688763 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1775f7-7fdb-4732-a86f-82de9e89b6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:33.703214 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-1a1775f7-7fdb-4732-a86f-82de9e89b6eb None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:33.704622 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-1a1775f7-7fdb-4732-a86f-82de9e89b6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:33.717838 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1031/2070] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:33.719431 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c894430-5f3e-4a65-800d-ab7e7e07a63b None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:33.720335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c894430-5f3e-4a65-800d-ab7e7e07a63b None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:33.720335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c894430-5f3e-4a65-800d-ab7e7e07a63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:33.750221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-1c894430-5f3e-4a65-800d-ab7e7e07a63b None None] MFA Rules not processed for user `acf77182176441ebb30e1e8c13c7f0da`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:33.751444 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-1c894430-5f3e-4a65-800d-ab7e7e07a63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:33.771661 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:33.781591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1040/2071] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:33.783311 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae52ce5-eb09-450a-b4aa-a6c2e17b96dc None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:33.783821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae52ce5-eb09-450a-b4aa-a6c2e17b96dc None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:33.784309 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae52ce5-eb09-450a-b4aa-a6c2e17b96dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:33.808233 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-eae52ce5-eb09-450a-b4aa-a6c2e17b96dc None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:33.810213 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-eae52ce5-eb09-450a-b4aa-a6c2e17b96dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:33.824315 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1041/2072] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:33.824315 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae52ce5-eb09-450a-b4aa-a6c2e17b96dc None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:33.825963 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-eae52ce5-eb09-450a-b4aa-a6c2e17b96dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:33.831227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1032/2073] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:33 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:33.831227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c894430-5f3e-4a65-800d-ab7e7e07a63b None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:33.846267 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c894430-5f3e-4a65-800d-ab7e7e07a63b None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:33.851688 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faccb62-87f8-4208-9ef4-88c6fadbbc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:33.853150 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faccb62-87f8-4208-9ef4-88c6fadbbc9e admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:33.853433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faccb62-87f8-4208-9ef4-88c6fadbbc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:33.853705 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faccb62-87f8-4208-9ef4-88c6fadbbc9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:33.854436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faccb62-87f8-4208-9ef4-88c6fadbbc9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:51:33.854745 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faccb62-87f8-4208-9ef4-88c6fadbbc9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:33.856582 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faccb62-87f8-4208-9ef4-88c6fadbbc9e admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:33.872024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1033/2074] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:51:33.872024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faccb62-87f8-4208-9ef4-88c6fadbbc9e admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:33.872378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-9faccb62-87f8-4208-9ef4-88c6fadbbc9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:33.885608 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:33.928851 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae52ce5-eb09-450a-b4aa-a6c2e17b96dc None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:33.932241 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faccb62-87f8-4208-9ef4-88c6fadbbc9e admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:33.933029 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30379c5-233b-4264-a6aa-3adea574fad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:33.933648 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30379c5-233b-4264-a6aa-3adea574fad6 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:33.933939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30379c5-233b-4264-a6aa-3adea574fad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:33.934222 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30379c5-233b-4264-a6aa-3adea574fad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:33.936346 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1bc905-b0c1-4cd5-9bdb-0c88b46ee3f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:33.937022 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bc905-b0c1-4cd5-9bdb-0c88b46ee3f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:33.937339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bc905-b0c1-4cd5-9bdb-0c88b46ee3f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:33.937621 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bc905-b0c1-4cd5-9bdb-0c88b46ee3f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:33.943746 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30379c5-233b-4264-a6aa-3adea574fad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:33.945289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30379c5-233b-4264-a6aa-3adea574fad6 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:33.948442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1bc905-b0c1-4cd5-9bdb-0c88b46ee3f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:33.949947 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1bc905-b0c1-4cd5-9bdb-0c88b46ee3f4 service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:33.955636 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1042/2075] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:33.956454 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30379c5-233b-4264-a6aa-3adea574fad6 service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:33.965086 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1034/2076] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:33 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:33.966778 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30379c5-233b-4264-a6aa-3adea574fad6 service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:33.976131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534354d-a532-4d50-8f66-d46dfe4ed169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:33.976131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534354d-a532-4d50-8f66-d46dfe4ed169 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:33.976131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534354d-a532-4d50-8f66-d46dfe4ed169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:33.976131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534354d-a532-4d50-8f66-d46dfe4ed169 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:33.976449 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1534354d-a532-4d50-8f66-d46dfe4ed169 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:33.976449 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1534354d-a532-4d50-8f66-d46dfe4ed169 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:34.099244 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1043/2077] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:51:33 2022] POST /identity/v3/projects => generated 365 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:34.103983 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1bc905-b0c1-4cd5-9bdb-0c88b46ee3f4 service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:34.114847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1bc905-b0c1-4cd5-9bdb-0c88b46ee3f4 service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:34.172120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534354d-a532-4d50-8f66-d46dfe4ed169 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:34.174726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c020c39-ee8c-4c0d-a406-69530ba7dd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:34.175597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c020c39-ee8c-4c0d-a406-69530ba7dd33 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:34.176242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c020c39-ee8c-4c0d-a406-69530ba7dd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:34.176572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c020c39-ee8c-4c0d-a406-69530ba7dd33 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:34.177306 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c020c39-ee8c-4c0d-a406-69530ba7dd33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:34.179317 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c020c39-ee8c-4c0d-a406-69530ba7dd33 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:34.227703 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534354d-a532-4d50-8f66-d46dfe4ed169 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:34.231398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2d95d-9f13-4968-9ff6-3972929555f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:34.232046 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2d95d-9f13-4968-9ff6-3972929555f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:34.232328 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2d95d-9f13-4968-9ff6-3972929555f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:34.232603 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2d95d-9f13-4968-9ff6-3972929555f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:34.234434 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-6aa2d95d-9f13-4968-9ff6-3972929555f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:34.261500 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1035/2078] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:51:34 2022] POST /identity/v3/users => generated 283 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:34.265396 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c020c39-ee8c-4c0d-a406-69530ba7dd33 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:34.295773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c020c39-ee8c-4c0d-a406-69530ba7dd33 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:34.308452 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d63a02-879b-46bb-805b-fd23db3464ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:34.309274 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d63a02-879b-46bb-805b-fd23db3464ee admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:34.309763 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d63a02-879b-46bb-805b-fd23db3464ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:34.310053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d63a02-879b-46bb-805b-fd23db3464ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:34.312298 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d63a02-879b-46bb-805b-fd23db3464ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:34.316225 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d63a02-879b-46bb-805b-fd23db3464ee admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:34.337148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1036/2079] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:51:34.338395 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2d95d-9f13-4968-9ff6-3972929555f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:34.339364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d63a02-879b-46bb-805b-fd23db3464ee admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:34.339961 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa2d95d-9f13-4968-9ff6-3972929555f7 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:34.352157 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d63a02-879b-46bb-805b-fd23db3464ee admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:34.353297 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1044/2080] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:34 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:51:34.364224 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71314a97-1e2e-4f3b-9422-48a7ef036295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:34.365474 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71314a97-1e2e-4f3b-9422-48a7ef036295 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:34.365474 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71314a97-1e2e-4f3b-9422-48a7ef036295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:34.365653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71314a97-1e2e-4f3b-9422-48a7ef036295 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:34.366628 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71314a97-1e2e-4f3b-9422-48a7ef036295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:34.369462 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71314a97-1e2e-4f3b-9422-48a7ef036295 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:34.384490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1037/2081] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:34.388663 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2d95d-9f13-4968-9ff6-3972929555f7 service cinder] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:34.405738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2d95d-9f13-4968-9ff6-3972929555f7 service cinder] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:34.410924 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1e2af-8f40-4c42-a140-36957761f567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:34.411826 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1e2af-8f40-4c42-a140-36957761f567 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:34.412105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1e2af-8f40-4c42-a140-36957761f567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:34.412381 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b1e2af-8f40-4c42-a140-36957761f567 admin admin] PATH_INFO: `/v3/projects/26987a6990bf4ace9c6674b250955f39/users/ba002a25cfaa47cbb738dc1470106c7d/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:34.428781 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b1e2af-8f40-4c42-a140-36957761f567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26987a6990bf4ace9c6674b250955f39, user_id=ba002a25cfaa47cbb738dc1470106c7d, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:34.433326 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b1e2af-8f40-4c42-a140-36957761f567 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:34.460737 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:34.565343 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1045/2082] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:34 2022] PUT /identity/v3/projects/26987a6990bf4ace9c6674b250955f39/users/ba002a25cfaa47cbb738dc1470106c7d/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:34.577608 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f2731-cde8-42cb-800b-fbb239878bee None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:34.577904 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f2731-cde8-42cb-800b-fbb239878bee None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:34.580800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f2731-cde8-42cb-800b-fbb239878bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:34.607957 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-546f2731-cde8-42cb-800b-fbb239878bee None None] MFA Rules not processed for user `ba002a25cfaa47cbb738dc1470106c7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:34.609099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-546f2731-cde8-42cb-800b-fbb239878bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:34.659094 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:34.677740 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1038/2083] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:34 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:34.689946 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1e2af-8f40-4c42-a140-36957761f567 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:34.691683 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-31b1e2af-8f40-4c42-a140-36957761f567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:34.771134 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b1e2af-8f40-4c42-a140-36957761f567 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:34.776334 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477e39c-21ab-4116-a6c7-1f67d76ad91b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:34.777350 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477e39c-21ab-4116-a6c7-1f67d76ad91b service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:34.777921 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477e39c-21ab-4116-a6c7-1f67d76ad91b service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:34.779015 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9477e39c-21ab-4116-a6c7-1f67d76ad91b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:34.790317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9477e39c-21ab-4116-a6c7-1f67d76ad91b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:34.792470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9477e39c-21ab-4116-a6c7-1f67d76ad91b service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:34.804914 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1046/2084] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:34 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:36.092326 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:36.292789 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:37.352974 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:37.994301 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:39.466442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:40.321811 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf514fc-6179-445b-94c2-0ada3cca319a None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:40.322568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf514fc-6179-445b-94c2-0ada3cca319a None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:40.322739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf514fc-6179-445b-94c2-0ada3cca319a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:40.365936 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-4bf514fc-6179-445b-94c2-0ada3cca319a None None] MFA Rules not processed for user `c0f12003099f4f80bb89c2a335d231ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:40.371403 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-4bf514fc-6179-445b-94c2-0ada3cca319a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:40.403585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:40.475841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1039/2085] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:40.480393 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477e39c-21ab-4116-a6c7-1f67d76ad91b service cinder] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:40.482035 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-9477e39c-21ab-4116-a6c7-1f67d76ad91b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:40.584758 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9477e39c-21ab-4116-a6c7-1f67d76ad91b service cinder] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:40.590635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee55e002-7780-4da1-a410-3d5848e7b642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:40.591499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55e002-7780-4da1-a410-3d5848e7b642 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:40.592180 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55e002-7780-4da1-a410-3d5848e7b642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:40.592616 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55e002-7780-4da1-a410-3d5848e7b642 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:40.593882 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee55e002-7780-4da1-a410-3d5848e7b642 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:40.595823 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee55e002-7780-4da1-a410-3d5848e7b642 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:40.712770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1047/2086] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:51:40 2022] POST /identity/v3/projects => generated 383 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:40.713172 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf514fc-6179-445b-94c2-0ada3cca319a None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:40.738211 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf514fc-6179-445b-94c2-0ada3cca319a None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:40.877141 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e095a2-1af4-4c16-bee4-ca753b6a9451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:40.878038 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e095a2-1af4-4c16-bee4-ca753b6a9451 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:40.878559 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e095a2-1af4-4c16-bee4-ca753b6a9451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:40.878804 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e095a2-1af4-4c16-bee4-ca753b6a9451 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:40.879622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e095a2-1af4-4c16-bee4-ca753b6a9451 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:40.882499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e095a2-1af4-4c16-bee4-ca753b6a9451 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:41.039880 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1040/2087] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:51:40 2022] POST /identity/v3/users => generated 298 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:51:41.043858 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee55e002-7780-4da1-a410-3d5848e7b642 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:41.056005 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee55e002-7780-4da1-a410-3d5848e7b642 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:41.063875 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd559b4c-2d0a-4fe7-8650-f9904c96a51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:41.064807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd559b4c-2d0a-4fe7-8650-f9904c96a51d admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:41.069108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd559b4c-2d0a-4fe7-8650-f9904c96a51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:41.069701 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd559b4c-2d0a-4fe7-8650-f9904c96a51d admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:41.073484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd559b4c-2d0a-4fe7-8650-f9904c96a51d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:41.075461 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd559b4c-2d0a-4fe7-8650-f9904c96a51d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:41.098069 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1048/2088] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:41.102907 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e095a2-1af4-4c16-bee4-ca753b6a9451 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:41.155477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e095a2-1af4-4c16-bee4-ca753b6a9451 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:41.165506 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8d310-f159-4e4f-8819-672abc386e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:41.166836 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8d310-f159-4e4f-8819-672abc386e6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:41.173662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8d310-f159-4e4f-8819-672abc386e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:41.173979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8d310-f159-4e4f-8819-672abc386e6b admin admin] PATH_INFO: `/v3/projects/8019c7f3ebd94943a22d342c2ba83aa6/users/a4327447184a413380ad998dadcab5d9/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:41.225659 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8d310-f159-4e4f-8819-672abc386e6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8019c7f3ebd94943a22d342c2ba83aa6, user_id=a4327447184a413380ad998dadcab5d9, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:41.229203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8d310-f159-4e4f-8819-672abc386e6b admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:41.344978 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1041/2089] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:41 2022] PUT /identity/v3/projects/8019c7f3ebd94943a22d342c2ba83aa6/users/a4327447184a413380ad998dadcab5d9/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:51:41.347709 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd559b4c-2d0a-4fe7-8650-f9904c96a51d admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:41.366156 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd559b4c-2d0a-4fe7-8650-f9904c96a51d admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:41.445897 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d1c6cf-9555-480d-9cd3-2f4c4dbfdb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:41.446721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1c6cf-9555-480d-9cd3-2f4c4dbfdb4c admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:41.447004 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1c6cf-9555-480d-9cd3-2f4c4dbfdb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:41.447603 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1c6cf-9555-480d-9cd3-2f4c4dbfdb4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:41.448482 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d1c6cf-9555-480d-9cd3-2f4c4dbfdb4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:41.450136 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d1c6cf-9555-480d-9cd3-2f4c4dbfdb4c admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:41.466630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1049/2090] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:41.467647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8d310-f159-4e4f-8819-672abc386e6b admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:41.489145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8d310-f159-4e4f-8819-672abc386e6b admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:41.505973 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f4ad22-3eef-4004-ac9d-4358a795dc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:41.506899 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4ad22-3eef-4004-ac9d-4358a795dc6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:41.507184 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4ad22-3eef-4004-ac9d-4358a795dc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:41.507461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f4ad22-3eef-4004-ac9d-4358a795dc6a admin admin] PATH_INFO: `/v3/projects/8019c7f3ebd94943a22d342c2ba83aa6/users/a4327447184a413380ad998dadcab5d9/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:41.517176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f4ad22-3eef-4004-ac9d-4358a795dc6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8019c7f3ebd94943a22d342c2ba83aa6, user_id=a4327447184a413380ad998dadcab5d9, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:41.520382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f4ad22-3eef-4004-ac9d-4358a795dc6a admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:41.549843 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1042/2091] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:41 2022] PUT /identity/v3/projects/8019c7f3ebd94943a22d342c2ba83aa6/users/a4327447184a413380ad998dadcab5d9/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:51:41.578340 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf01d3-f673-45dd-a49e-230faaeff4fe None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:41.579416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf01d3-f673-45dd-a49e-230faaeff4fe None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:41.579871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf01d3-f673-45dd-a49e-230faaeff4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:41.614427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-bbbf01d3-f673-45dd-a49e-230faaeff4fe None None] MFA Rules not processed for user `a13d2c50c5204c859872e99bc45c6566`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:41.616151 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-bbbf01d3-f673-45dd-a49e-230faaeff4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:41.673042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:41.692884 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1050/2092] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:41.703096 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e6014-f1da-48b1-8793-da4521af32bb None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:41.703389 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e6014-f1da-48b1-8793-da4521af32bb None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:41.703672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e6014-f1da-48b1-8793-da4521af32bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:41.741090 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-8d2e6014-f1da-48b1-8793-da4521af32bb None None] MFA Rules not processed for user `a13d2c50c5204c859872e99bc45c6566`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:41.743243 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-8d2e6014-f1da-48b1-8793-da4521af32bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:41.781958 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1043/2093] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:41.791727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf01d3-f673-45dd-a49e-230faaeff4fe None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:41.793200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-bbbf01d3-f673-45dd-a49e-230faaeff4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:41.888393 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf01d3-f673-45dd-a49e-230faaeff4fe None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:41.899273 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755b792-d9ed-4497-8e3d-74cc2c9feda7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:41.899963 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755b792-d9ed-4497-8e3d-74cc2c9feda7 service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:41.900251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755b792-d9ed-4497-8e3d-74cc2c9feda7 service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:41.900533 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755b792-d9ed-4497-8e3d-74cc2c9feda7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:41.911263 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755b792-d9ed-4497-8e3d-74cc2c9feda7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:41.913104 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755b792-d9ed-4497-8e3d-74cc2c9feda7 service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:41.924663 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1051/2094] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:42.352575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e6014-f1da-48b1-8793-da4521af32bb None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:42.354169 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-8d2e6014-f1da-48b1-8793-da4521af32bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:42.548888 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e6014-f1da-48b1-8793-da4521af32bb None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:42.567685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2851a-f8b9-45f1-8bc4-8a00ec36165e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:42.570554 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2851a-f8b9-45f1-8bc4-8a00ec36165e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:42.571089 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2851a-f8b9-45f1-8bc4-8a00ec36165e admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:42.573264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2851a-f8b9-45f1-8bc4-8a00ec36165e admin admin] PATH_INFO: `/v3/users/13ddfaf9ad514026bfb589a2ba494e27` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:42.573264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2851a-f8b9-45f1-8bc4-8a00ec36165e admin admin] RBAC: Authorizing `identity:delete_user(user_id=13ddfaf9ad514026bfb589a2ba494e27)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:42.575575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2851a-f8b9-45f1-8bc4-8a00ec36165e admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:42.595884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:42.850735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c2b2851a-f8b9-45f1-8bc4-8a00ec36165e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13ddfaf9ad514026bfb589a2ba494e27'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:42.898854 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c2b2851a-f8b9-45f1-8bc4-8a00ec36165e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13ddfaf9ad514026bfb589a2ba494e27'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:43.016256 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c2b2851a-f8b9-45f1-8bc4-8a00ec36165e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13ddfaf9ad514026bfb589a2ba494e27'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:43.017050 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c2b2851a-f8b9-45f1-8bc4-8a00ec36165e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13ddfaf9ad514026bfb589a2ba494e27'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:43.017677 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c2b2851a-f8b9-45f1-8bc4-8a00ec36165e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13ddfaf9ad514026bfb589a2ba494e27'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:43.048857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1044/2095] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:42 2022] DELETE /identity/v3/users/13ddfaf9ad514026bfb589a2ba494e27 => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:43.053771 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43476394-5a64-4c85-a96d-df62d78aebaa None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:43.055036 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43476394-5a64-4c85-a96d-df62d78aebaa None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:43.055495 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43476394-5a64-4c85-a96d-df62d78aebaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:43.100151 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-43476394-5a64-4c85-a96d-df62d78aebaa None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:43.102395 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-43476394-5a64-4c85-a96d-df62d78aebaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:43.272593 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1052/2096] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:43.298654 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2851a-f8b9-45f1-8bc4-8a00ec36165e admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:43.300419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-c2b2851a-f8b9-45f1-8bc4-8a00ec36165e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:43.414553 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2851a-f8b9-45f1-8bc4-8a00ec36165e admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:43.416748 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea93132-22c5-4292-86c7-78cc4b59265c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:43.417627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea93132-22c5-4292-86c7-78cc4b59265c service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:43.417627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea93132-22c5-4292-86c7-78cc4b59265c service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:43.418382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea93132-22c5-4292-86c7-78cc4b59265c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:43.433570 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea93132-22c5-4292-86c7-78cc4b59265c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:43.435377 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea93132-22c5-4292-86c7-78cc4b59265c service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:43.452854 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1045/2097] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:43.949718 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411968ec-d0eb-4880-8633-e1d1f7f5f36b None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:43.950207 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411968ec-d0eb-4880-8633-e1d1f7f5f36b None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:43.950898 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411968ec-d0eb-4880-8633-e1d1f7f5f36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:43.994983 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-411968ec-d0eb-4880-8633-e1d1f7f5f36b None None] MFA Rules not processed for user `a13d2c50c5204c859872e99bc45c6566`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:43.996250 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-411968ec-d0eb-4880-8633-e1d1f7f5f36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:44.147198 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1053/2098] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:44.342018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea93132-22c5-4292-86c7-78cc4b59265c service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:44.345013 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-aea93132-22c5-4292-86c7-78cc4b59265c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:44.382897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea93132-22c5-4292-86c7-78cc4b59265c service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:44.389895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a6468-b9b4-4636-925c-3f4ec08c793b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:44.393712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a6468-b9b4-4636-925c-3f4ec08c793b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:44.393712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a6468-b9b4-4636-925c-3f4ec08c793b admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:44.393712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a6468-b9b4-4636-925c-3f4ec08c793b admin admin] PATH_INFO: `/v3/projects/851046691d294b1592f08f509bfedbbc` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:44.393712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a6468-b9b4-4636-925c-3f4ec08c793b admin admin] RBAC: Authorizing `identity:delete_project(project_id=851046691d294b1592f08f509bfedbbc)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:44.395552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a6468-b9b4-4636-925c-3f4ec08c793b admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:44.572202 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1f5a6468-b9b4-4636-925c-3f4ec08c793b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '851046691d294b1592f08f509bfedbbc', 'request_id': 'req-1f5a6468-b9b4-4636-925c-3f4ec08c793b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:44.592050 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1f5a6468-b9b4-4636-925c-3f4ec08c793b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '851046691d294b1592f08f509bfedbbc', 'request_id': 'req-1f5a6468-b9b4-4636-925c-3f4ec08c793b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:44.712946 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1f5a6468-b9b4-4636-925c-3f4ec08c793b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '851046691d294b1592f08f509bfedbbc', 'request_id': 'req-1f5a6468-b9b4-4636-925c-3f4ec08c793b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:44.736593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1f5a6468-b9b4-4636-925c-3f4ec08c793b admin admin] The token cache is being invalidate because project 851046691d294b1592f08f509bfedbbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:51:44.737946 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1f5a6468-b9b4-4636-925c-3f4ec08c793b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:44.738893 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1f5a6468-b9b4-4636-925c-3f4ec08c793b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:44.741191 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1046/2099] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:44 2022] DELETE /identity/v3/projects/851046691d294b1592f08f509bfedbbc => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:44.746725 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411968ec-d0eb-4880-8633-e1d1f7f5f36b None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:44.797762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411968ec-d0eb-4880-8633-e1d1f7f5f36b None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:44.828428 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:44.943052 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5064c23-2186-43a5-a094-625f36294d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:44.944827 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5064c23-2186-43a5-a094-625f36294d07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:44.944827 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5064c23-2186-43a5-a094-625f36294d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:44.944827 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5064c23-2186-43a5-a094-625f36294d07 admin admin] PATH_INFO: `/v3/users/08b0b01529924b469f0be9fa3047cad7` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:44.946562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5064c23-2186-43a5-a094-625f36294d07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08b0b01529924b469f0be9fa3047cad7)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:44.949952 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5064c23-2186-43a5-a094-625f36294d07 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:45.215461 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-a5064c23-2186-43a5-a094-625f36294d07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08b0b01529924b469f0be9fa3047cad7'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:45.231936 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-a5064c23-2186-43a5-a094-625f36294d07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08b0b01529924b469f0be9fa3047cad7'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:45.315995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-a5064c23-2186-43a5-a094-625f36294d07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08b0b01529924b469f0be9fa3047cad7'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:45.316987 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-a5064c23-2186-43a5-a094-625f36294d07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08b0b01529924b469f0be9fa3047cad7'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:45.335003 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-a5064c23-2186-43a5-a094-625f36294d07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08b0b01529924b469f0be9fa3047cad7'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:45.359283 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1054/2100] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:44 2022] DELETE /identity/v3/users/08b0b01529924b469f0be9fa3047cad7 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:46.047447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a6468-b9b4-4636-925c-3f4ec08c793b admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:46.048930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-1f5a6468-b9b4-4636-925c-3f4ec08c793b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:46.077054 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a6468-b9b4-4636-925c-3f4ec08c793b admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:46.099920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:46.237689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33331b06-fa1e-49c1-adce-80d80d318451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:46.238591 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33331b06-fa1e-49c1-adce-80d80d318451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:46.240024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33331b06-fa1e-49c1-adce-80d80d318451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:46.240024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33331b06-fa1e-49c1-adce-80d80d318451 admin admin] PATH_INFO: `/v3/users/0b73c2ab337040aba08c94a5da60849a` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:46.240834 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33331b06-fa1e-49c1-adce-80d80d318451 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b73c2ab337040aba08c94a5da60849a)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:46.243409 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33331b06-fa1e-49c1-adce-80d80d318451 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:46.274545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5064c23-2186-43a5-a094-625f36294d07 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:46.275706 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-a5064c23-2186-43a5-a094-625f36294d07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:46.298124 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5064c23-2186-43a5-a094-625f36294d07 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:46.322076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6405df82-72c0-4889-9d3a-0d1c39b46ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:46.323171 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405df82-72c0-4889-9d3a-0d1c39b46ac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:46.323625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405df82-72c0-4889-9d3a-0d1c39b46ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:46.324048 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6405df82-72c0-4889-9d3a-0d1c39b46ac1 admin admin] PATH_INFO: `/v3/projects/798ce34345de4bf2b12d41b4dc6c6e53` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:46.326664 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405df82-72c0-4889-9d3a-0d1c39b46ac1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=798ce34345de4bf2b12d41b4dc6c6e53)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:46.331557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6405df82-72c0-4889-9d3a-0d1c39b46ac1 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:46.569682 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-33331b06-fa1e-49c1-adce-80d80d318451 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b73c2ab337040aba08c94a5da60849a'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:46.576646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-6405df82-72c0-4889-9d3a-0d1c39b46ac1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '798ce34345de4bf2b12d41b4dc6c6e53', 'request_id': 'req-6405df82-72c0-4889-9d3a-0d1c39b46ac1'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:46.610054 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-6405df82-72c0-4889-9d3a-0d1c39b46ac1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '798ce34345de4bf2b12d41b4dc6c6e53', 'request_id': 'req-6405df82-72c0-4889-9d3a-0d1c39b46ac1'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:46.624633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-33331b06-fa1e-49c1-adce-80d80d318451 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b73c2ab337040aba08c94a5da60849a'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:46.624736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-6405df82-72c0-4889-9d3a-0d1c39b46ac1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '798ce34345de4bf2b12d41b4dc6c6e53', 'request_id': 'req-6405df82-72c0-4889-9d3a-0d1c39b46ac1'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:46.751995 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-33331b06-fa1e-49c1-adce-80d80d318451 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b73c2ab337040aba08c94a5da60849a'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:46.752885 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-33331b06-fa1e-49c1-adce-80d80d318451 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b73c2ab337040aba08c94a5da60849a'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:46.753688 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-33331b06-fa1e-49c1-adce-80d80d318451 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b73c2ab337040aba08c94a5da60849a'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:46.782786 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1047/2101] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:46 2022] DELETE /identity/v3/users/0b73c2ab337040aba08c94a5da60849a => generated 0 bytes in 733 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:46.784414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558332a5-f2b5-42ad-80f5-c7613fa5f79a None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:46.784710 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558332a5-f2b5-42ad-80f5-c7613fa5f79a None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:46.785009 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558332a5-f2b5-42ad-80f5-c7613fa5f79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:46.806781 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-558332a5-f2b5-42ad-80f5-c7613fa5f79a None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:46.809143 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-558332a5-f2b5-42ad-80f5-c7613fa5f79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:46.849333 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-6405df82-72c0-4889-9d3a-0d1c39b46ac1 admin admin] The token cache is being invalidate because project 798ce34345de4bf2b12d41b4dc6c6e53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:51:46.849633 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-6405df82-72c0-4889-9d3a-0d1c39b46ac1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:46.850449 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-6405df82-72c0-4889-9d3a-0d1c39b46ac1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:46.857006 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1055/2102] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:46 2022] DELETE /identity/v3/projects/798ce34345de4bf2b12d41b4dc6c6e53 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:51:46.860077 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f519b-40c2-4b95-9e93-83c2d09ef835 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:46.862375 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f519b-40c2-4b95-9e93-83c2d09ef835 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:46.862716 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717f519b-40c2-4b95-9e93-83c2d09ef835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:46.904554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-717f519b-40c2-4b95-9e93-83c2d09ef835 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:46.906725 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-717f519b-40c2-4b95-9e93-83c2d09ef835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:46.943608 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1048/2103] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:46.944546 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:46.958600 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558332a5-f2b5-42ad-80f5-c7613fa5f79a None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:46.960394 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-558332a5-f2b5-42ad-80f5-c7613fa5f79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:47.033957 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1056/2104] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:47.042853 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986a95f-794e-422e-97a3-5f17d7648949 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:47.043732 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986a95f-794e-422e-97a3-5f17d7648949 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:47.045172 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986a95f-794e-422e-97a3-5f17d7648949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:47.063002 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558332a5-f2b5-42ad-80f5-c7613fa5f79a None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:47.068693 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550cc901-a238-4fb0-8029-b559002ed976 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:47.069830 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cc901-a238-4fb0-8029-b559002ed976 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:47.071374 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cc901-a238-4fb0-8029-b559002ed976 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:47.072059 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cc901-a238-4fb0-8029-b559002ed976 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:47.085452 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550cc901-a238-4fb0-8029-b559002ed976 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:47.088189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550cc901-a238-4fb0-8029-b559002ed976 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:47.099524 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-0986a95f-794e-422e-97a3-5f17d7648949 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:47.102748 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-0986a95f-794e-422e-97a3-5f17d7648949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:47.104618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1049/2105] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:47.113739 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1057/2106] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:47.119576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550cc901-a238-4fb0-8029-b559002ed976 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:47.132162 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550cc901-a238-4fb0-8029-b559002ed976 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:47.138361 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2fe0cd-cce5-4fec-99a7-902e44d87165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:47.139400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fe0cd-cce5-4fec-99a7-902e44d87165 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:47.139908 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fe0cd-cce5-4fec-99a7-902e44d87165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:47.140618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2fe0cd-cce5-4fec-99a7-902e44d87165 admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:47.143664 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2fe0cd-cce5-4fec-99a7-902e44d87165 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:51:47.144285 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2fe0cd-cce5-4fec-99a7-902e44d87165 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:47.146168 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2fe0cd-cce5-4fec-99a7-902e44d87165 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:47.170161 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1050/2107] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:51:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:51:47.175013 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986a95f-794e-422e-97a3-5f17d7648949 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:47.190759 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986a95f-794e-422e-97a3-5f17d7648949 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:47.201045 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90569d12-ecbf-4344-b50e-03584e41da04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:47.203274 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90569d12-ecbf-4344-b50e-03584e41da04 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:47.203548 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90569d12-ecbf-4344-b50e-03584e41da04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:47.204707 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90569d12-ecbf-4344-b50e-03584e41da04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:47.205482 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90569d12-ecbf-4344-b50e-03584e41da04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:47.207624 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90569d12-ecbf-4344-b50e-03584e41da04 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:47.334021 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1058/2108] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:51:47 2022] POST /identity/v3/projects => generated 369 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:47.338544 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2fe0cd-cce5-4fec-99a7-902e44d87165 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:47.360691 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2fe0cd-cce5-4fec-99a7-902e44d87165 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:47.458268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa9af0-8978-4048-a10d-7ad0a359ac35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:47.459008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa9af0-8978-4048-a10d-7ad0a359ac35 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:47.463020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa9af0-8978-4048-a10d-7ad0a359ac35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:47.463020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa9af0-8978-4048-a10d-7ad0a359ac35 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:47.463554 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa9af0-8978-4048-a10d-7ad0a359ac35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:47.465578 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fa9af0-8978-4048-a10d-7ad0a359ac35 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:47.611190 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1051/2109] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:51:47 2022] POST /identity/v3/users => generated 285 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:47.615251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90569d12-ecbf-4344-b50e-03584e41da04 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:47.641604 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90569d12-ecbf-4344-b50e-03584e41da04 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:47.667224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b62d21-6fff-45ac-92b4-24fc357e4413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:47.667778 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b62d21-6fff-45ac-92b4-24fc357e4413 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:47.668066 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b62d21-6fff-45ac-92b4-24fc357e4413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:47.668358 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b62d21-6fff-45ac-92b4-24fc357e4413 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:47.669182 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b62d21-6fff-45ac-92b4-24fc357e4413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:47.682268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b62d21-6fff-45ac-92b4-24fc357e4413 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:47.706796 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1059/2110] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:47.708718 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa9af0-8978-4048-a10d-7ad0a359ac35 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:47.721822 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa9af0-8978-4048-a10d-7ad0a359ac35 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:47.753669 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094abdb-d927-404a-b91d-fcfb087b9da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:47.754373 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094abdb-d927-404a-b91d-fcfb087b9da8 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:47.754664 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094abdb-d927-404a-b91d-fcfb087b9da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:47.754952 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094abdb-d927-404a-b91d-fcfb087b9da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:47.759702 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094abdb-d927-404a-b91d-fcfb087b9da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:47.762902 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f094abdb-d927-404a-b91d-fcfb087b9da8 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:47.786084 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1052/2111] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:47 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:47.796193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b62d21-6fff-45ac-92b4-24fc357e4413 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:47.798324 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-92b62d21-6fff-45ac-92b4-24fc357e4413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:47.806105 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094abdb-d927-404a-b91d-fcfb087b9da8 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:47.820570 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094abdb-d927-404a-b91d-fcfb087b9da8 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:47.830327 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b62d21-6fff-45ac-92b4-24fc357e4413 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:47.834155 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2661367e-22bb-43f2-9fea-a5ef8ad2319b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:47.836181 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661367e-22bb-43f2-9fea-a5ef8ad2319b admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:47.836886 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661367e-22bb-43f2-9fea-a5ef8ad2319b admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:47.842352 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2661367e-22bb-43f2-9fea-a5ef8ad2319b admin admin] PATH_INFO: `/v3/projects/77f7ff4e73e843a6884ec6e06c556232/users/0b6e44b901ba469f9997ae3e189b0e00/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:47.845845 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e17424-483c-4345-8c13-0897fa434321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:47.848244 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e17424-483c-4345-8c13-0897fa434321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:47.852376 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e17424-483c-4345-8c13-0897fa434321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:47.853005 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e17424-483c-4345-8c13-0897fa434321 admin admin] PATH_INFO: `/v3/projects/05d4942fb11647f1b0dba5331a6399f7` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:47.856835 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e17424-483c-4345-8c13-0897fa434321 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05d4942fb11647f1b0dba5331a6399f7)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:47.860588 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e17424-483c-4345-8c13-0897fa434321 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:47.878564 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2661367e-22bb-43f2-9fea-a5ef8ad2319b admin admin] RBAC: Authorizing `identity:create_grant(project_id=77f7ff4e73e843a6884ec6e06c556232, user_id=0b6e44b901ba469f9997ae3e189b0e00, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:47.884171 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2661367e-22bb-43f2-9fea-a5ef8ad2319b admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:48.074087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1053/2112] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:47 2022] PUT /identity/v3/projects/77f7ff4e73e843a6884ec6e06c556232/users/0b6e44b901ba469f9997ae3e189b0e00/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:48.100709 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-94e17424-483c-4345-8c13-0897fa434321 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05d4942fb11647f1b0dba5331a6399f7', 'request_id': 'req-94e17424-483c-4345-8c13-0897fa434321'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:48.102626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50f8c9-fdaf-460c-a7fc-f3a09a57f496 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:48.105418 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50f8c9-fdaf-460c-a7fc-f3a09a57f496 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:48.108464 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50f8c9-fdaf-460c-a7fc-f3a09a57f496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:48.113337 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-94e17424-483c-4345-8c13-0897fa434321 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05d4942fb11647f1b0dba5331a6399f7', 'request_id': 'req-94e17424-483c-4345-8c13-0897fa434321'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:48.125325 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-94e17424-483c-4345-8c13-0897fa434321 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05d4942fb11647f1b0dba5331a6399f7', 'request_id': 'req-94e17424-483c-4345-8c13-0897fa434321'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:48.184156 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-ce50f8c9-fdaf-460c-a7fc-f3a09a57f496 None None] MFA Rules not processed for user `0b6e44b901ba469f9997ae3e189b0e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:48.186034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-ce50f8c9-fdaf-460c-a7fc-f3a09a57f496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:48.250387 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-94e17424-483c-4345-8c13-0897fa434321 admin admin] The token cache is being invalidate because project 05d4942fb11647f1b0dba5331a6399f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:51:48.250847 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-94e17424-483c-4345-8c13-0897fa434321 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:48.252006 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-94e17424-483c-4345-8c13-0897fa434321 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:48.256975 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1060/2113] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:47 2022] DELETE /identity/v3/projects/05d4942fb11647f1b0dba5331a6399f7 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:48.259663 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c5df7-f768-4abf-b120-33a5b800ba5e None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:48.259961 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c5df7-f768-4abf-b120-33a5b800ba5e None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:48.260238 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c5df7-f768-4abf-b120-33a5b800ba5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:48.302628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-df7c5df7-f768-4abf-b120-33a5b800ba5e None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:48.305240 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-df7c5df7-f768-4abf-b120-33a5b800ba5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:48.315946 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1054/2114] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:48 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:48.317097 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50f8c9-fdaf-460c-a7fc-f3a09a57f496 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:48.318608 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-ce50f8c9-fdaf-460c-a7fc-f3a09a57f496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:48.384996 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1061/2115] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:51:48.410141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b800461-86a6-40e2-aaaa-01cf9b7c114e None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:48.410432 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b800461-86a6-40e2-aaaa-01cf9b7c114e None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:48.410717 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b800461-86a6-40e2-aaaa-01cf9b7c114e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:48.433442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50f8c9-fdaf-460c-a7fc-f3a09a57f496 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:48.447160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595748b9-e266-43ae-90e1-6d74d986ddb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:48.447994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595748b9-e266-43ae-90e1-6d74d986ddb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:48.448383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595748b9-e266-43ae-90e1-6d74d986ddb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:48.449142 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595748b9-e266-43ae-90e1-6d74d986ddb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:48.449855 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595748b9-e266-43ae-90e1-6d74d986ddb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:48.452009 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595748b9-e266-43ae-90e1-6d74d986ddb4 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:48.457834 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-3b800461-86a6-40e2-aaaa-01cf9b7c114e None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:48.463858 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-3b800461-86a6-40e2-aaaa-01cf9b7c114e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:48.489146 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1062/2116] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:48.493818 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b800461-86a6-40e2-aaaa-01cf9b7c114e None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:48.551759 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b800461-86a6-40e2-aaaa-01cf9b7c114e None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:48.567587 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581aab8-da35-407c-bf7d-0db05959c9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:48.568630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581aab8-da35-407c-bf7d-0db05959c9f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:48.569899 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581aab8-da35-407c-bf7d-0db05959c9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:48.571304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581aab8-da35-407c-bf7d-0db05959c9f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:48.573730 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581aab8-da35-407c-bf7d-0db05959c9f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:51:48.574536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581aab8-da35-407c-bf7d-0db05959c9f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:48.580451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0581aab8-da35-407c-bf7d-0db05959c9f1 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:48.612379 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1063/2117] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:51:48.616291 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581aab8-da35-407c-bf7d-0db05959c9f1 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:48.644030 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0581aab8-da35-407c-bf7d-0db05959c9f1 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:48.669247 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5e56a-4a7e-4404-899e-ebba0d02b223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:48.670059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5e56a-4a7e-4404-899e-ebba0d02b223 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:48.670527 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5e56a-4a7e-4404-899e-ebba0d02b223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:48.670819 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a5e56a-4a7e-4404-899e-ebba0d02b223 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:48.672008 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a5e56a-4a7e-4404-899e-ebba0d02b223 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:48.680983 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1055/2118] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:51:48 2022] POST /identity/v3/projects => generated 367 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:48.681199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a5e56a-4a7e-4404-899e-ebba0d02b223 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:48.684528 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595748b9-e266-43ae-90e1-6d74d986ddb4 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:48.818897 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1064/2119] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:51:48 2022] POST /identity/v3/projects => generated 373 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:48.824031 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5e56a-4a7e-4404-899e-ebba0d02b223 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:48.825938 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595748b9-e266-43ae-90e1-6d74d986ddb4 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:48.987864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7586ed3-ba6c-4a49-850a-d68083e1b362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:48.988483 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7586ed3-ba6c-4a49-850a-d68083e1b362 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:48.988767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7586ed3-ba6c-4a49-850a-d68083e1b362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:48.989068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7586ed3-ba6c-4a49-850a-d68083e1b362 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:48.989831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7586ed3-ba6c-4a49-850a-d68083e1b362 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:48.991993 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7586ed3-ba6c-4a49-850a-d68083e1b362 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:48.997553 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a5e56a-4a7e-4404-899e-ebba0d02b223 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:49.027873 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588d31c-fd37-4e62-b03d-d3bfe0a85f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:49.028645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588d31c-fd37-4e62-b03d-d3bfe0a85f7f admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:49.028932 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588d31c-fd37-4e62-b03d-d3bfe0a85f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:49.029239 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588d31c-fd37-4e62-b03d-d3bfe0a85f7f admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:49.029966 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9588d31c-fd37-4e62-b03d-d3bfe0a85f7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:49.031965 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9588d31c-fd37-4e62-b03d-d3bfe0a85f7f admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:49.088130 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:49.165280 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1056/2120] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:51:48 2022] POST /identity/v3/users => generated 290 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:49.168058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7586ed3-ba6c-4a49-850a-d68083e1b362 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:49.183079 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7586ed3-ba6c-4a49-850a-d68083e1b362 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:49.192387 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2209e-3dee-4707-91d7-7a65ecf6c233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:49.192387 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2209e-3dee-4707-91d7-7a65ecf6c233 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:49.192387 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2209e-3dee-4707-91d7-7a65ecf6c233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:49.192387 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2209e-3dee-4707-91d7-7a65ecf6c233 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:49.193250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2209e-3dee-4707-91d7-7a65ecf6c233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:49.197350 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1065/2121] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:51:48 2022] POST /identity/v3/users => generated 287 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:49.198418 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588d31c-fd37-4e62-b03d-d3bfe0a85f7f admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:49.199030 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2209e-3dee-4707-91d7-7a65ecf6c233 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:49.214781 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1057/2122] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:49 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:49.215647 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588d31c-fd37-4e62-b03d-d3bfe0a85f7f admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:49.216138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2209e-3dee-4707-91d7-7a65ecf6c233 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:49.225460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a777e-3192-4743-8de4-4695c326cc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:49.226839 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a777e-3192-4743-8de4-4695c326cc46 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:49.228210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a777e-3192-4743-8de4-4695c326cc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:49.228210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117a777e-3192-4743-8de4-4695c326cc46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:49.228421 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117a777e-3192-4743-8de4-4695c326cc46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:49.230142 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117a777e-3192-4743-8de4-4695c326cc46 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:49.238066 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2209e-3dee-4707-91d7-7a65ecf6c233 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:49.254876 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300f2bd-42e5-4cb9-a93d-f1a661445279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:49.256090 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300f2bd-42e5-4cb9-a93d-f1a661445279 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:49.256539 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300f2bd-42e5-4cb9-a93d-f1a661445279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:49.259618 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1066/2123] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:49 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:51:49.260226 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300f2bd-42e5-4cb9-a93d-f1a661445279 admin admin] PATH_INFO: `/v3/projects/62a9704cc0a3433dab02429784d6ee5d/users/43bfcc05f8a3408d90fe5cd24a968cb2/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:49.260771 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a777e-3192-4743-8de4-4695c326cc46 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:49.278838 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117a777e-3192-4743-8de4-4695c326cc46 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:49.285698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300f2bd-42e5-4cb9-a93d-f1a661445279 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a9704cc0a3433dab02429784d6ee5d, user_id=43bfcc05f8a3408d90fe5cd24a968cb2, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:49.289257 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c300f2bd-42e5-4cb9-a93d-f1a661445279 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:49.291930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f417e-dd0a-4763-b13e-93e83e8356ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:49.292634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f417e-dd0a-4763-b13e-93e83e8356ba admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:49.292979 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f417e-dd0a-4763-b13e-93e83e8356ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:49.293307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f417e-dd0a-4763-b13e-93e83e8356ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:49.294213 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f417e-dd0a-4763-b13e-93e83e8356ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:49.295708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f417e-dd0a-4763-b13e-93e83e8356ba admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:49.328023 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1067/2124] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:49 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:49.328023 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f417e-dd0a-4763-b13e-93e83e8356ba admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:49.355446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f417e-dd0a-4763-b13e-93e83e8356ba admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:49.369507 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160ad82f-d12d-4b40-b76d-e91aee68d078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:49.372114 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ad82f-d12d-4b40-b76d-e91aee68d078 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:49.372114 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ad82f-d12d-4b40-b76d-e91aee68d078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:49.372114 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160ad82f-d12d-4b40-b76d-e91aee68d078 admin admin] PATH_INFO: `/v3/projects/246fa1f940e846f8b389b1351c4156f1/users/5185a726047c4e5e814a995beb41f2c9/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:49.373729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1058/2125] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:49 2022] PUT /identity/v3/projects/62a9704cc0a3433dab02429784d6ee5d/users/43bfcc05f8a3408d90fe5cd24a968cb2/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:49.381521 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300f2bd-42e5-4cb9-a93d-f1a661445279 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:49.408986 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ad82f-d12d-4b40-b76d-e91aee68d078 admin admin] RBAC: Authorizing `identity:create_grant(project_id=246fa1f940e846f8b389b1351c4156f1, user_id=5185a726047c4e5e814a995beb41f2c9, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:49.412397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160ad82f-d12d-4b40-b76d-e91aee68d078 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:49.531534 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1068/2126] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:49 2022] PUT /identity/v3/projects/246fa1f940e846f8b389b1351c4156f1/users/5185a726047c4e5e814a995beb41f2c9/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:49.533814 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba13cbe-7555-432d-99a8-757b02907063 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:49.534233 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba13cbe-7555-432d-99a8-757b02907063 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:49.534624 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba13cbe-7555-432d-99a8-757b02907063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:49.538078 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300f2bd-42e5-4cb9-a93d-f1a661445279 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:49.614849 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17747206-37b8-4c4d-ae97-d66fe61b6d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:49.614849 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17747206-37b8-4c4d-ae97-d66fe61b6d93 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:49.614849 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17747206-37b8-4c4d-ae97-d66fe61b6d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:49.614849 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17747206-37b8-4c4d-ae97-d66fe61b6d93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:49.616446 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17747206-37b8-4c4d-ae97-d66fe61b6d93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:49.617569 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17747206-37b8-4c4d-ae97-d66fe61b6d93 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:49.625509 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-8ba13cbe-7555-432d-99a8-757b02907063 None None] MFA Rules not processed for user `5185a726047c4e5e814a995beb41f2c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:49.627044 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-8ba13cbe-7555-432d-99a8-757b02907063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:49.631929 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1059/2127] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:49 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:49.634041 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17747206-37b8-4c4d-ae97-d66fe61b6d93 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:49.655623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17747206-37b8-4c4d-ae97-d66fe61b6d93 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:49.686566 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8428fe29-fb98-4bb9-bbe3-2e244373d69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:49.694097 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428fe29-fb98-4bb9-bbe3-2e244373d69c admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:49.694426 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428fe29-fb98-4bb9-bbe3-2e244373d69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:49.694706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8428fe29-fb98-4bb9-bbe3-2e244373d69c admin admin] PATH_INFO: `/v3/projects/62a9704cc0a3433dab02429784d6ee5d/users/43bfcc05f8a3408d90fe5cd24a968cb2/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:49.704237 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428fe29-fb98-4bb9-bbe3-2e244373d69c admin admin] RBAC: Authorizing `identity:create_grant(project_id=62a9704cc0a3433dab02429784d6ee5d, user_id=43bfcc05f8a3408d90fe5cd24a968cb2, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:49.714790 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8428fe29-fb98-4bb9-bbe3-2e244373d69c admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:49.759269 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1060/2128] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:49 2022] PUT /identity/v3/projects/62a9704cc0a3433dab02429784d6ee5d/users/43bfcc05f8a3408d90fe5cd24a968cb2/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:49.762369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd7d4d-df85-4c47-a8a8-3220aad1e635 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:49.766280 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd7d4d-df85-4c47-a8a8-3220aad1e635 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:49.766760 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd7d4d-df85-4c47-a8a8-3220aad1e635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:49.793247 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1069/2129] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:49 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:51:49.796195 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba13cbe-7555-432d-99a8-757b02907063 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:49.814934 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba13cbe-7555-432d-99a8-757b02907063 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:49.816978 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-7cfd7d4d-df85-4c47-a8a8-3220aad1e635 None None] MFA Rules not processed for user `43bfcc05f8a3408d90fe5cd24a968cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:49.819468 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-7cfd7d4d-df85-4c47-a8a8-3220aad1e635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:49.959392 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e3c91-9ef7-41ca-bfd7-eb7bc483b2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:49.961878 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e3c91-9ef7-41ca-bfd7-eb7bc483b2ee admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:49.961878 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e3c91-9ef7-41ca-bfd7-eb7bc483b2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:49.961878 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e3c91-9ef7-41ca-bfd7-eb7bc483b2ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:49.962815 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e3c91-9ef7-41ca-bfd7-eb7bc483b2ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:49.971877 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e3c91-9ef7-41ca-bfd7-eb7bc483b2ee admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:49.989660 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:49.995685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1061/2130] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:49 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:50.008327 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd7d4d-df85-4c47-a8a8-3220aad1e635 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:50.023581 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-7cfd7d4d-df85-4c47-a8a8-3220aad1e635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:50.128861 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd7d4d-df85-4c47-a8a8-3220aad1e635 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:50.131436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1070/2131] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:51:49 2022] POST /identity/v3/projects => generated 375 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:51:50.136070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e3c91-9ef7-41ca-bfd7-eb7bc483b2ee admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:50.194416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842deff-aa80-4614-ab8c-df0e137d552b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:50.196525 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842deff-aa80-4614-ab8c-df0e137d552b service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:50.197136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842deff-aa80-4614-ab8c-df0e137d552b service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:50.197936 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842deff-aa80-4614-ab8c-df0e137d552b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:50.306179 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e3c91-9ef7-41ca-bfd7-eb7bc483b2ee admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:50.318913 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b466e1-07d8-4378-816e-b47a694dc555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:50.326207 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b466e1-07d8-4378-816e-b47a694dc555 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:50.326207 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b466e1-07d8-4378-816e-b47a694dc555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:50.326207 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b466e1-07d8-4378-816e-b47a694dc555 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:50.326207 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b466e1-07d8-4378-816e-b47a694dc555 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:50.328962 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b466e1-07d8-4378-816e-b47a694dc555 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:50.389271 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842deff-aa80-4614-ab8c-df0e137d552b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:50.391105 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842deff-aa80-4614-ab8c-df0e137d552b service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:50.415291 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1062/2132] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:50 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:51:50.485668 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1071/2133] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:51:50 2022] POST /identity/v3/users => generated 294 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:50.489571 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842deff-aa80-4614-ab8c-df0e137d552b service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:50.511748 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842deff-aa80-4614-ab8c-df0e137d552b service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:50.519418 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fcb3e2-23f8-46bc-8c53-d6be8c6a8792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:50.524382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcb3e2-23f8-46bc-8c53-d6be8c6a8792 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:50.524382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcb3e2-23f8-46bc-8c53-d6be8c6a8792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:50.524382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fcb3e2-23f8-46bc-8c53-d6be8c6a8792 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:50.524382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fcb3e2-23f8-46bc-8c53-d6be8c6a8792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:50.524382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fcb3e2-23f8-46bc-8c53-d6be8c6a8792 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:50.537780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1063/2134] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:50.542355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b466e1-07d8-4378-816e-b47a694dc555 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:50.553794 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b466e1-07d8-4378-816e-b47a694dc555 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:50.576853 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea888b-2c5f-4e77-b2d8-50b6cbaeb7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:50.577830 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea888b-2c5f-4e77-b2d8-50b6cbaeb7b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:50.578252 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea888b-2c5f-4e77-b2d8-50b6cbaeb7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:50.578727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fea888b-2c5f-4e77-b2d8-50b6cbaeb7b4 admin admin] PATH_INFO: `/v3/projects/91988bc3a29d4d92a1f5f9576512b62d/users/7185388f206f4289b94182610dc7274e/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:50.616050 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea888b-2c5f-4e77-b2d8-50b6cbaeb7b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91988bc3a29d4d92a1f5f9576512b62d, user_id=7185388f206f4289b94182610dc7274e, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:50.621711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fea888b-2c5f-4e77-b2d8-50b6cbaeb7b4 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:50.775642 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1072/2135] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:50 2022] PUT /identity/v3/projects/91988bc3a29d4d92a1f5f9576512b62d/users/7185388f206f4289b94182610dc7274e/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:50.780106 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fcb3e2-23f8-46bc-8c53-d6be8c6a8792 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:50.795201 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fcb3e2-23f8-46bc-8c53-d6be8c6a8792 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:50.927262 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f9e86-4f54-422b-b281-a99660e918a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:50.928023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f9e86-4f54-422b-b281-a99660e918a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:50.928317 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f9e86-4f54-422b-b281-a99660e918a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:50.928649 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f9e86-4f54-422b-b281-a99660e918a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:50.929487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f9e86-4f54-422b-b281-a99660e918a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:50.931247 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f9e86-4f54-422b-b281-a99660e918a6 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:50.994735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1064/2136] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:50.997853 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea888b-2c5f-4e77-b2d8-50b6cbaeb7b4 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:51.017428 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fea888b-2c5f-4e77-b2d8-50b6cbaeb7b4 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:51.043982 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd94876-983e-43de-8866-0badcff45b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:51.047936 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd94876-983e-43de-8866-0badcff45b3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:51.048436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd94876-983e-43de-8866-0badcff45b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:51.048738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd94876-983e-43de-8866-0badcff45b3c admin admin] PATH_INFO: `/v3/projects/91988bc3a29d4d92a1f5f9576512b62d/users/7185388f206f4289b94182610dc7274e/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:51.061771 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd94876-983e-43de-8866-0badcff45b3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=91988bc3a29d4d92a1f5f9576512b62d, user_id=7185388f206f4289b94182610dc7274e, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:51.061771 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd94876-983e-43de-8866-0badcff45b3c admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:51.097607 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1073/2137] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:50 2022] PUT /identity/v3/projects/91988bc3a29d4d92a1f5f9576512b62d/users/7185388f206f4289b94182610dc7274e/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:51.109652 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a69f5e-4b7c-40d1-943b-7632fd4d1d1e None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:51.110069 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a69f5e-4b7c-40d1-943b-7632fd4d1d1e None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:51.110883 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a69f5e-4b7c-40d1-943b-7632fd4d1d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:51.166440 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-d9a69f5e-4b7c-40d1-943b-7632fd4d1d1e None None] MFA Rules not processed for user `7185388f206f4289b94182610dc7274e`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:51.168613 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d9a69f5e-4b7c-40d1-943b-7632fd4d1d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:51.280335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1065/2138] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:51 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:51.346240 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd94876-983e-43de-8866-0badcff45b3c admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:51.449157 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd94876-983e-43de-8866-0badcff45b3c admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:51.454921 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f3b7c-66a1-4276-8d12-d96701932365 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:51.455712 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f3b7c-66a1-4276-8d12-d96701932365 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:51.456527 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f3b7c-66a1-4276-8d12-d96701932365 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:51.456875 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f3b7c-66a1-4276-8d12-d96701932365 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:51.514161 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a69f5e-4b7c-40d1-943b-7632fd4d1d1e None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:51.524421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d9a69f5e-4b7c-40d1-943b-7632fd4d1d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:51.579867 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f3b7c-66a1-4276-8d12-d96701932365 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:51.581575 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f3b7c-66a1-4276-8d12-d96701932365 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:51.617642 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1074/2139] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:51 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:51.678782 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a69f5e-4b7c-40d1-943b-7632fd4d1d1e None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:51.684094 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de94d2-c33c-4965-8096-ce5d017fe3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:51.685302 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de94d2-c33c-4965-8096-ce5d017fe3bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:51.686444 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de94d2-c33c-4965-8096-ce5d017fe3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:51.686827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70de94d2-c33c-4965-8096-ce5d017fe3bb admin admin] PATH_INFO: `/v3/users/2b58d2c0879a46af818c44d4f34a1ece` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:51.697752 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de94d2-c33c-4965-8096-ce5d017fe3bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b58d2c0879a46af818c44d4f34a1ece)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:51.704412 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70de94d2-c33c-4965-8096-ce5d017fe3bb admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:52.001721 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-70de94d2-c33c-4965-8096-ce5d017fe3bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b58d2c0879a46af818c44d4f34a1ece'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:52.017439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:52.033398 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-70de94d2-c33c-4965-8096-ce5d017fe3bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b58d2c0879a46af818c44d4f34a1ece'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:52.151306 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-70de94d2-c33c-4965-8096-ce5d017fe3bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b58d2c0879a46af818c44d4f34a1ece'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:52.151985 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-70de94d2-c33c-4965-8096-ce5d017fe3bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b58d2c0879a46af818c44d4f34a1ece'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:52.152858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-70de94d2-c33c-4965-8096-ce5d017fe3bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b58d2c0879a46af818c44d4f34a1ece'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:52.173780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1066/2140] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:51 2022] DELETE /identity/v3/users/2b58d2c0879a46af818c44d4f34a1ece => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:52.178253 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fb20c-6172-467c-89c9-a0fa3b579e66 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:52.178619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fb20c-6172-467c-89c9-a0fa3b579e66 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:52.178812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6fb20c-6172-467c-89c9-a0fa3b579e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:52.217464 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-8e6fb20c-6172-467c-89c9-a0fa3b579e66 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:52.218946 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-8e6fb20c-6172-467c-89c9-a0fa3b579e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:52.365966 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1075/2141] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:52.379968 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de94d2-c33c-4965-8096-ce5d017fe3bb admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:52.381926 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-70de94d2-c33c-4965-8096-ce5d017fe3bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:52.470602 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70de94d2-c33c-4965-8096-ce5d017fe3bb admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:52.486289 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19ac29-3fa6-4c0e-b194-7c24f53365ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:52.491188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19ac29-3fa6-4c0e-b194-7c24f53365ec service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:52.491503 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19ac29-3fa6-4c0e-b194-7c24f53365ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:52.492322 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19ac29-3fa6-4c0e-b194-7c24f53365ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:52.503664 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19ac29-3fa6-4c0e-b194-7c24f53365ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:52.512487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19ac29-3fa6-4c0e-b194-7c24f53365ec service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:52.557603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1067/2142] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:52.747703 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:53.544829 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6fb20c-6172-467c-89c9-a0fa3b579e66 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:53.546550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-8e6fb20c-6172-467c-89c9-a0fa3b579e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:53.576009 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6fb20c-6172-467c-89c9-a0fa3b579e66 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:53.583947 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ed8da-49fc-4e4e-8a2b-fb22f140c87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:53.586142 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ed8da-49fc-4e4e-8a2b-fb22f140c87f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:53.586793 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ed8da-49fc-4e4e-8a2b-fb22f140c87f admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:53.587250 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ed8da-49fc-4e4e-8a2b-fb22f140c87f admin admin] PATH_INFO: `/v3/projects/51efa3774bfa4adaafbf26a0bb090690` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:53.590230 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ed8da-49fc-4e4e-8a2b-fb22f140c87f admin admin] RBAC: Authorizing `identity:delete_project(project_id=51efa3774bfa4adaafbf26a0bb090690)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:53.594088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ed8da-49fc-4e4e-8a2b-fb22f140c87f admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:53.723234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-733ed8da-49fc-4e4e-8a2b-fb22f140c87f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51efa3774bfa4adaafbf26a0bb090690', 'request_id': 'req-733ed8da-49fc-4e4e-8a2b-fb22f140c87f'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:53.731223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-733ed8da-49fc-4e4e-8a2b-fb22f140c87f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51efa3774bfa4adaafbf26a0bb090690', 'request_id': 'req-733ed8da-49fc-4e4e-8a2b-fb22f140c87f'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:53.740281 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-733ed8da-49fc-4e4e-8a2b-fb22f140c87f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51efa3774bfa4adaafbf26a0bb090690', 'request_id': 'req-733ed8da-49fc-4e4e-8a2b-fb22f140c87f'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:53.742666 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:53.751623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104a194-6241-4267-a72f-fe84ba8d5c93 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:53.751915 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104a194-6241-4267-a72f-fe84ba8d5c93 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:53.752204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104a194-6241-4267-a72f-fe84ba8d5c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:53.794742 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-e104a194-6241-4267-a72f-fe84ba8d5c93 None None] MFA Rules not processed for user `a4327447184a413380ad998dadcab5d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:53.798217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-e104a194-6241-4267-a72f-fe84ba8d5c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:53.836903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-733ed8da-49fc-4e4e-8a2b-fb22f140c87f admin admin] The token cache is being invalidate because project 51efa3774bfa4adaafbf26a0bb090690 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:51:53.837441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-733ed8da-49fc-4e4e-8a2b-fb22f140c87f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:53.839417 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-733ed8da-49fc-4e4e-8a2b-fb22f140c87f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:53.842920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1076/2143] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:53 2022] DELETE /identity/v3/projects/51efa3774bfa4adaafbf26a0bb090690 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:53.849836 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ed8da-49fc-4e4e-8a2b-fb22f140c87f admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:53.851308 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-733ed8da-49fc-4e4e-8a2b-fb22f140c87f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:53.887101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1068/2144] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:51:53.898672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104a194-6241-4267-a72f-fe84ba8d5c93 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:53.902025 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-e104a194-6241-4267-a72f-fe84ba8d5c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:53.945852 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ed8da-49fc-4e4e-8a2b-fb22f140c87f admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:53.951051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f9120-03b4-4f9d-bfea-f8346370af9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:53.952014 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f9120-03b4-4f9d-bfea-f8346370af9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:53.952373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f9120-03b4-4f9d-bfea-f8346370af9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:53.952732 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f9120-03b4-4f9d-bfea-f8346370af9d admin admin] PATH_INFO: `/v3/users/acf77182176441ebb30e1e8c13c7f0da` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:53.953988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f9120-03b4-4f9d-bfea-f8346370af9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=acf77182176441ebb30e1e8c13c7f0da)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:53.955967 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f9120-03b4-4f9d-bfea-f8346370af9d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:53.994835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104a194-6241-4267-a72f-fe84ba8d5c93 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:53.999809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bab3e0-0ffa-4356-b143-6794bf165263 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1347c0bd4ada4139ad194d423958be6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:54.001383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bab3e0-0ffa-4356-b143-6794bf165263 service glance] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:54.001717 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bab3e0-0ffa-4356-b143-6794bf165263 service glance] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:54.002003 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bab3e0-0ffa-4356-b143-6794bf165263 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:54.004456 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-45bab3e0-0ffa-4356-b143-6794bf165263 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:54.130202 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bab3e0-0ffa-4356-b143-6794bf165263 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:54.131768 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bab3e0-0ffa-4356-b143-6794bf165263 service glance] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:54.156255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1069/2145] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:54.175419 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-6b9f9120-03b4-4f9d-bfea-f8346370af9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acf77182176441ebb30e1e8c13c7f0da'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:54.200276 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-6b9f9120-03b4-4f9d-bfea-f8346370af9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acf77182176441ebb30e1e8c13c7f0da'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:54.309192 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:54.315086 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-6b9f9120-03b4-4f9d-bfea-f8346370af9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acf77182176441ebb30e1e8c13c7f0da'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:54.315930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-6b9f9120-03b4-4f9d-bfea-f8346370af9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acf77182176441ebb30e1e8c13c7f0da'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:54.333601 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-6b9f9120-03b4-4f9d-bfea-f8346370af9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acf77182176441ebb30e1e8c13c7f0da'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:54.348625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1077/2146] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:53 2022] DELETE /identity/v3/users/acf77182176441ebb30e1e8c13c7f0da => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:54.528878 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaaccd8-0df5-443d-aab3-b48bc4d6fe40 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:54.529254 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaaccd8-0df5-443d-aab3-b48bc4d6fe40 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:54.529831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaaccd8-0df5-443d-aab3-b48bc4d6fe40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:54.583206 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-faaaccd8-0df5-443d-aab3-b48bc4d6fe40 None None] MFA Rules not processed for user `c700b0adb9f54947a88c1baa86accb9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:54.584902 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-faaaccd8-0df5-443d-aab3-b48bc4d6fe40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:54.686173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1070/2147] 10.222.0.13 () {58 vars in 1009 bytes} [Fri May 27 19:51:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:54.750188 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f9120-03b4-4f9d-bfea-f8346370af9d admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:54.752550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-6b9f9120-03b4-4f9d-bfea-f8346370af9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:54.874707 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f9120-03b4-4f9d-bfea-f8346370af9d admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:54.893578 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde80aee-edd2-41fc-a409-46b27d320467 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37bc6d0abb454de19e1ab2f46bf4a593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:54.893578 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde80aee-edd2-41fc-a409-46b27d320467 service swift] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:54.893578 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde80aee-edd2-41fc-a409-46b27d320467 service swift] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:54.893578 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde80aee-edd2-41fc-a409-46b27d320467 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:54.917565 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde80aee-edd2-41fc-a409-46b27d320467 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:54.919582 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde80aee-edd2-41fc-a409-46b27d320467 service swift] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:54.925185 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaaccd8-0df5-443d-aab3-b48bc4d6fe40 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:54.926599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-faaaccd8-0df5-443d-aab3-b48bc4d6fe40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:54.945730 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1078/2148] 10.222.0.13 () {60 vars in 1384 bytes} [Fri May 27 19:51:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 19:51:55.080877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaaccd8-0df5-443d-aab3-b48bc4d6fe40 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:55.094239 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc9478e-2874-4ac4-bfb6-b90fc5750505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:55.095136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9478e-2874-4ac4-bfb6-b90fc5750505 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:55.095420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9478e-2874-4ac4-bfb6-b90fc5750505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:55.095699 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc9478e-2874-4ac4-bfb6-b90fc5750505 admin admin] PATH_INFO: `/v3/users/0b6e44b901ba469f9997ae3e189b0e00` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:55.098588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9478e-2874-4ac4-bfb6-b90fc5750505 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b6e44b901ba469f9997ae3e189b0e00)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:55.100689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc9478e-2874-4ac4-bfb6-b90fc5750505 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:55.252253 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde80aee-edd2-41fc-a409-46b27d320467 service swift] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:55.253984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-bde80aee-edd2-41fc-a409-46b27d320467 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:55.285197 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde80aee-edd2-41fc-a409-46b27d320467 service swift] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:55.298545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e531b9-12c7-40ec-8296-907d0b0054fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:55.303518 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e531b9-12c7-40ec-8296-907d0b0054fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:55.307066 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e531b9-12c7-40ec-8296-907d0b0054fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:55.307363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e531b9-12c7-40ec-8296-907d0b0054fa admin admin] PATH_INFO: `/v3/projects/30f12cc1e28c4da295dbe1b8168e3b27` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:55.311203 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e531b9-12c7-40ec-8296-907d0b0054fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=30f12cc1e28c4da295dbe1b8168e3b27)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:55.320649 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e531b9-12c7-40ec-8296-907d0b0054fa admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:55.337786 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-bbc9478e-2874-4ac4-bfb6-b90fc5750505 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b6e44b901ba469f9997ae3e189b0e00'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:55.380287 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-bbc9478e-2874-4ac4-bfb6-b90fc5750505 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b6e44b901ba469f9997ae3e189b0e00'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:55.465221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-bbc9478e-2874-4ac4-bfb6-b90fc5750505 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b6e44b901ba469f9997ae3e189b0e00'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:55.469558 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-bbc9478e-2874-4ac4-bfb6-b90fc5750505 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b6e44b901ba469f9997ae3e189b0e00'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:55.469558 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-bbc9478e-2874-4ac4-bfb6-b90fc5750505 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b6e44b901ba469f9997ae3e189b0e00'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:55.500299 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1071/2149] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:54 2022] DELETE /identity/v3/users/0b6e44b901ba469f9997ae3e189b0e00 => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:51:55.505613 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca97455-2487-4a68-b9f5-f22761047ec5 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:55.506203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca97455-2487-4a68-b9f5-f22761047ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:55.506689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca97455-2487-4a68-b9f5-f22761047ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:55.536822 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-cca97455-2487-4a68-b9f5-f22761047ec5 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:55.539484 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-cca97455-2487-4a68-b9f5-f22761047ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:55.561474 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-40e531b9-12c7-40ec-8296-907d0b0054fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30f12cc1e28c4da295dbe1b8168e3b27', 'request_id': 'req-40e531b9-12c7-40ec-8296-907d0b0054fa'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:55.570839 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-40e531b9-12c7-40ec-8296-907d0b0054fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30f12cc1e28c4da295dbe1b8168e3b27', 'request_id': 'req-40e531b9-12c7-40ec-8296-907d0b0054fa'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:55.583927 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-40e531b9-12c7-40ec-8296-907d0b0054fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30f12cc1e28c4da295dbe1b8168e3b27', 'request_id': 'req-40e531b9-12c7-40ec-8296-907d0b0054fa'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:55.584477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:55.622135 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1072/2150] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:55.634189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca97455-2487-4a68-b9f5-f22761047ec5 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:55.635962 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-cca97455-2487-4a68-b9f5-f22761047ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:55.697092 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-40e531b9-12c7-40ec-8296-907d0b0054fa admin admin] The token cache is being invalidate because project 30f12cc1e28c4da295dbe1b8168e3b27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:51:55.697597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-40e531b9-12c7-40ec-8296-907d0b0054fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:55.699539 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-40e531b9-12c7-40ec-8296-907d0b0054fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:55.702781 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1079/2151] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:55 2022] DELETE /identity/v3/projects/30f12cc1e28c4da295dbe1b8168e3b27 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:51:55.708373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61d6b8-6185-4cb2-aa89-2bad50af8ba9 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:55.708878 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61d6b8-6185-4cb2-aa89-2bad50af8ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:55.709610 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61d6b8-6185-4cb2-aa89-2bad50af8ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:55.765977 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-bb61d6b8-6185-4cb2-aa89-2bad50af8ba9 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:55.769629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-bb61d6b8-6185-4cb2-aa89-2bad50af8ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:55.782546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca97455-2487-4a68-b9f5-f22761047ec5 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:55.797450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1080/2152] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:55.800097 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4798da6-7130-4017-87be-30698c8ecbe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:55.803038 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c39e7-8313-4575-ad4d-a73d0ee68a0e None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:55.803456 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4798da6-7130-4017-87be-30698c8ecbe8 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:55.803955 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4798da6-7130-4017-87be-30698c8ecbe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:55.804384 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c39e7-8313-4575-ad4d-a73d0ee68a0e None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:55.804805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4798da6-7130-4017-87be-30698c8ecbe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:55.805991 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c39e7-8313-4575-ad4d-a73d0ee68a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:55.811298 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-e4798da6-7130-4017-87be-30698c8ecbe8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:55.849884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4798da6-7130-4017-87be-30698c8ecbe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:55.853734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4798da6-7130-4017-87be-30698c8ecbe8 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:55.855318 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-b85c39e7-8313-4575-ad4d-a73d0ee68a0e None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:55.865474 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-b85c39e7-8313-4575-ad4d-a73d0ee68a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:55.890854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1081/2153] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:55.892645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1073/2154] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:55.903892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c39e7-8313-4575-ad4d-a73d0ee68a0e None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:55.990888 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c39e7-8313-4575-ad4d-a73d0ee68a0e None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:55.990888 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d58e16-615c-4c5a-a9a0-a9bd8666baac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:55.990888 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d58e16-615c-4c5a-a9a0-a9bd8666baac admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:55.990888 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d58e16-615c-4c5a-a9a0-a9bd8666baac admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:55.991241 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d58e16-615c-4c5a-a9a0-a9bd8666baac admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:55.991241 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d58e16-615c-4c5a-a9a0-a9bd8666baac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:51:55.991241 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d58e16-615c-4c5a-a9a0-a9bd8666baac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:55.991241 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d58e16-615c-4c5a-a9a0-a9bd8666baac admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:55.991241 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1082/2155] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:51:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:51:55.994569 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4798da6-7130-4017-87be-30698c8ecbe8 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:56.011827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4798da6-7130-4017-87be-30698c8ecbe8 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:56.039716 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e63ab7-7e7c-4a14-8696-8145382d008b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:56.041075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e63ab7-7e7c-4a14-8696-8145382d008b admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:56.042778 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e63ab7-7e7c-4a14-8696-8145382d008b admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:56.043240 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e63ab7-7e7c-4a14-8696-8145382d008b admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:56.044138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e63ab7-7e7c-4a14-8696-8145382d008b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:56.050315 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e63ab7-7e7c-4a14-8696-8145382d008b admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:56.249597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1074/2156] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:51:55 2022] POST /identity/v3/projects => generated 361 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:56.257997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d58e16-615c-4c5a-a9a0-a9bd8666baac admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:56.274535 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d58e16-615c-4c5a-a9a0-a9bd8666baac admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:56.402546 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae6e1a-f3c9-4bed-9b3f-5be9d59907a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:56.403330 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae6e1a-f3c9-4bed-9b3f-5be9d59907a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:56.403711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae6e1a-f3c9-4bed-9b3f-5be9d59907a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:56.404085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae6e1a-f3c9-4bed-9b3f-5be9d59907a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:56.404876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae6e1a-f3c9-4bed-9b3f-5be9d59907a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:56.407449 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae6e1a-f3c9-4bed-9b3f-5be9d59907a0 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:56.534633 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1083/2157] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:51:56 2022] POST /identity/v3/users => generated 281 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:51:56.539722 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e63ab7-7e7c-4a14-8696-8145382d008b admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:56.565967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e63ab7-7e7c-4a14-8696-8145382d008b admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:56.584676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1168337-becf-40c7-8515-5f1fb7da2de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:56.584676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1168337-becf-40c7-8515-5f1fb7da2de6 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:56.584676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1168337-becf-40c7-8515-5f1fb7da2de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:56.584676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1168337-becf-40c7-8515-5f1fb7da2de6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:56.588557 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1168337-becf-40c7-8515-5f1fb7da2de6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:56.590788 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1168337-becf-40c7-8515-5f1fb7da2de6 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:56.615361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae6e1a-f3c9-4bed-9b3f-5be9d59907a0 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:56.619467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1075/2158] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:51:56.621465 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1168337-becf-40c7-8515-5f1fb7da2de6 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:56.623231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-8dae6e1a-f3c9-4bed-9b3f-5be9d59907a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:56.634518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1168337-becf-40c7-8515-5f1fb7da2de6 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:56.644566 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7450274-e700-407c-a238-07f2c66aa6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:56.645531 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7450274-e700-407c-a238-07f2c66aa6dc admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:56.645989 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7450274-e700-407c-a238-07f2c66aa6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:56.647265 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7450274-e700-407c-a238-07f2c66aa6dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:56.649296 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7450274-e700-407c-a238-07f2c66aa6dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:56.656459 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7450274-e700-407c-a238-07f2c66aa6dc admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:56.667970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae6e1a-f3c9-4bed-9b3f-5be9d59907a0 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:56.675174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1076/2159] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:51:56 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:51:56.685005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7450274-e700-407c-a238-07f2c66aa6dc admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:56.711422 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f750d0be-9d2b-4aab-b216-600493ead543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:56.711624 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750d0be-9d2b-4aab-b216-600493ead543 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:56.711950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750d0be-9d2b-4aab-b216-600493ead543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:56.712251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750d0be-9d2b-4aab-b216-600493ead543 admin admin] PATH_INFO: `/v3/users/5185a726047c4e5e814a995beb41f2c9` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:56.714412 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f750d0be-9d2b-4aab-b216-600493ead543 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5185a726047c4e5e814a995beb41f2c9)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:56.717059 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7450274-e700-407c-a238-07f2c66aa6dc admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:56.717821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:56.718646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f750d0be-9d2b-4aab-b216-600493ead543 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:56.736400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b165817-a104-477e-9e8c-8c44f9cf9b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:56.737978 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b165817-a104-477e-9e8c-8c44f9cf9b3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:56.745117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b165817-a104-477e-9e8c-8c44f9cf9b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:56.745117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b165817-a104-477e-9e8c-8c44f9cf9b3d admin admin] PATH_INFO: `/v3/projects/7a241839e9674489a93a734a6c44c2e0/users/038179e8be02476fb9b73ba3db11875d/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:56.774280 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b165817-a104-477e-9e8c-8c44f9cf9b3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a241839e9674489a93a734a6c44c2e0, user_id=038179e8be02476fb9b73ba3db11875d, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:56.777773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b165817-a104-477e-9e8c-8c44f9cf9b3d admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:56.949382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1077/2160] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:51:56 2022] PUT /identity/v3/projects/7a241839e9674489a93a734a6c44c2e0/users/038179e8be02476fb9b73ba3db11875d/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:56.950430 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b165817-a104-477e-9e8c-8c44f9cf9b3d admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:56.961413 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-2b165817-a104-477e-9e8c-8c44f9cf9b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:57.010471 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f750d0be-9d2b-4aab-b216-600493ead543 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5185a726047c4e5e814a995beb41f2c9'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:57.042259 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f750d0be-9d2b-4aab-b216-600493ead543 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5185a726047c4e5e814a995beb41f2c9'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:57.089459 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b165817-a104-477e-9e8c-8c44f9cf9b3d admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:57.095831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64445b-795f-414c-821d-2ffe01e43240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:57.096348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64445b-795f-414c-821d-2ffe01e43240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:57.096641 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64445b-795f-414c-821d-2ffe01e43240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:57.096928 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64445b-795f-414c-821d-2ffe01e43240 admin admin] PATH_INFO: `/v3/projects/77f7ff4e73e843a6884ec6e06c556232` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:57.098233 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64445b-795f-414c-821d-2ffe01e43240 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77f7ff4e73e843a6884ec6e06c556232)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:57.100357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64445b-795f-414c-821d-2ffe01e43240 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:57.123348 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f750d0be-9d2b-4aab-b216-600493ead543 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5185a726047c4e5e814a995beb41f2c9'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:57.124020 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f750d0be-9d2b-4aab-b216-600493ead543 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5185a726047c4e5e814a995beb41f2c9'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:57.141872 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f750d0be-9d2b-4aab-b216-600493ead543 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5185a726047c4e5e814a995beb41f2c9'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:57.163998 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1084/2161] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:56 2022] DELETE /identity/v3/users/5185a726047c4e5e814a995beb41f2c9 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:57.166398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e15bf-feb2-4ab8-a978-f06d481137cc None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:57.166398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e15bf-feb2-4ab8-a978-f06d481137cc None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:57.166398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e15bf-feb2-4ab8-a978-f06d481137cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:57.209425 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-219e15bf-feb2-4ab8-a978-f06d481137cc None None] MFA Rules not processed for user `038179e8be02476fb9b73ba3db11875d`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:57.211732 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-219e15bf-feb2-4ab8-a978-f06d481137cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:57.331424 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-bd64445b-795f-414c-821d-2ffe01e43240 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77f7ff4e73e843a6884ec6e06c556232', 'request_id': 'req-bd64445b-795f-414c-821d-2ffe01e43240'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:57.339395 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-bd64445b-795f-414c-821d-2ffe01e43240 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77f7ff4e73e843a6884ec6e06c556232', 'request_id': 'req-bd64445b-795f-414c-821d-2ffe01e43240'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:57.379148 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1085/2162] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:57.379148 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6867a73-2fe4-4f90-8622-f3d88bbe33ee None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:57.379148 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6867a73-2fe4-4f90-8622-f3d88bbe33ee None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:57.398172 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6867a73-2fe4-4f90-8622-f3d88bbe33ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:57.439088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-e6867a73-2fe4-4f90-8622-f3d88bbe33ee None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:57.441353 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-e6867a73-2fe4-4f90-8622-f3d88bbe33ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:57.445620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-bd64445b-795f-414c-821d-2ffe01e43240 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77f7ff4e73e843a6884ec6e06c556232', 'request_id': 'req-bd64445b-795f-414c-821d-2ffe01e43240'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:57.473640 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-bd64445b-795f-414c-821d-2ffe01e43240 admin admin] The token cache is being invalidate because project 77f7ff4e73e843a6884ec6e06c556232 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:51:57.473934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-bd64445b-795f-414c-821d-2ffe01e43240 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:57.479168 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-bd64445b-795f-414c-821d-2ffe01e43240 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:57.491049 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1078/2163] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:56 2022] DELETE /identity/v3/projects/77f7ff4e73e843a6884ec6e06c556232 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:57.492188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64445b-795f-414c-821d-2ffe01e43240 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:57.498604 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-bd64445b-795f-414c-821d-2ffe01e43240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:57.640944 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1086/2164] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:57.642147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6867a73-2fe4-4f90-8622-f3d88bbe33ee None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:57.644461 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-e6867a73-2fe4-4f90-8622-f3d88bbe33ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:57.667943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64445b-795f-414c-821d-2ffe01e43240 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:57.675943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4cf263-5221-4ee6-8072-4894a47a234a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:57.680358 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4cf263-5221-4ee6-8072-4894a47a234a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:57.680358 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4cf263-5221-4ee6-8072-4894a47a234a admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:57.680358 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4cf263-5221-4ee6-8072-4894a47a234a admin admin] PATH_INFO: `/v3/users/43bfcc05f8a3408d90fe5cd24a968cb2` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:57.680358 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4cf263-5221-4ee6-8072-4894a47a234a admin admin] RBAC: Authorizing `identity:delete_user(user_id=43bfcc05f8a3408d90fe5cd24a968cb2)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:57.682580 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4cf263-5221-4ee6-8072-4894a47a234a admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:57.805660 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6867a73-2fe4-4f90-8622-f3d88bbe33ee None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:57.809956 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264fa6d-b007-423f-a75a-e9e7b1c81c24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:57.810841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264fa6d-b007-423f-a75a-e9e7b1c81c24 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:57.811248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264fa6d-b007-423f-a75a-e9e7b1c81c24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:57.811625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264fa6d-b007-423f-a75a-e9e7b1c81c24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:57.814470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-f264fa6d-b007-423f-a75a-e9e7b1c81c24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:57.937411 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1b4cf263-5221-4ee6-8072-4894a47a234a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43bfcc05f8a3408d90fe5cd24a968cb2'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:58.003771 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264fa6d-b007-423f-a75a-e9e7b1c81c24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:58.008612 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1b4cf263-5221-4ee6-8072-4894a47a234a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43bfcc05f8a3408d90fe5cd24a968cb2'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:58.011665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264fa6d-b007-423f-a75a-e9e7b1c81c24 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:58.052894 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1087/2165] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:58.056193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264fa6d-b007-423f-a75a-e9e7b1c81c24 service cinder] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:58.061132 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-f264fa6d-b007-423f-a75a-e9e7b1c81c24 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:58.082928 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1b4cf263-5221-4ee6-8072-4894a47a234a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43bfcc05f8a3408d90fe5cd24a968cb2'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:58.083580 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1b4cf263-5221-4ee6-8072-4894a47a234a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43bfcc05f8a3408d90fe5cd24a968cb2'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:58.084185 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1b4cf263-5221-4ee6-8072-4894a47a234a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43bfcc05f8a3408d90fe5cd24a968cb2'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:58.121456 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1079/2166] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:57 2022] DELETE /identity/v3/users/43bfcc05f8a3408d90fe5cd24a968cb2 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:58.129922 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4cf263-5221-4ee6-8072-4894a47a234a admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:58.132872 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-1b4cf263-5221-4ee6-8072-4894a47a234a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:58.210472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264fa6d-b007-423f-a75a-e9e7b1c81c24 service cinder] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:58.402379 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4cf263-5221-4ee6-8072-4894a47a234a admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:58.417953 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ee238-3afb-49f9-9f67-f4219293c135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:58.418085 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67545063-665b-44bc-9338-82ce1b7037dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:58.418653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67545063-665b-44bc-9338-82ce1b7037dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:58.419040 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67545063-665b-44bc-9338-82ce1b7037dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:58.419420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67545063-665b-44bc-9338-82ce1b7037dd admin admin] PATH_INFO: `/v3/users/038179e8be02476fb9b73ba3db11875d` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:58.421632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67545063-665b-44bc-9338-82ce1b7037dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=038179e8be02476fb9b73ba3db11875d)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:58.423733 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67545063-665b-44bc-9338-82ce1b7037dd admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:58.427589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ee238-3afb-49f9-9f67-f4219293c135 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:58.428149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ee238-3afb-49f9-9f67-f4219293c135 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:58.428615 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ee238-3afb-49f9-9f67-f4219293c135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:58.438824 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-994ee238-3afb-49f9-9f67-f4219293c135 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:58.478685 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ee238-3afb-49f9-9f67-f4219293c135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:58.483504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ee238-3afb-49f9-9f67-f4219293c135 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:58.495923 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1088/2167] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:58.731374 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-67545063-665b-44bc-9338-82ce1b7037dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '038179e8be02476fb9b73ba3db11875d'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:58.773188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-67545063-665b-44bc-9338-82ce1b7037dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '038179e8be02476fb9b73ba3db11875d'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:58.904842 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-67545063-665b-44bc-9338-82ce1b7037dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '038179e8be02476fb9b73ba3db11875d'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:58.907467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-67545063-665b-44bc-9338-82ce1b7037dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '038179e8be02476fb9b73ba3db11875d'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:58.907467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-67545063-665b-44bc-9338-82ce1b7037dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '038179e8be02476fb9b73ba3db11875d'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:58.909198 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:51:58.958093 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1080/2168] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:51:58 2022] DELETE /identity/v3/users/038179e8be02476fb9b73ba3db11875d => generated 0 bytes in 836 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:51:58.964301 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9a857-c38f-4d2e-a6fb-635356883a0d None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:58.964944 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9a857-c38f-4d2e-a6fb-635356883a0d None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:58.965405 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9a857-c38f-4d2e-a6fb-635356883a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:59.018459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-caa9a857-c38f-4d2e-a6fb-635356883a0d None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:59.019588 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-caa9a857-c38f-4d2e-a6fb-635356883a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:59.150071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1089/2169] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:51:59.189740 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67545063-665b-44bc-9338-82ce1b7037dd admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:59.193145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-67545063-665b-44bc-9338-82ce1b7037dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:59.339284 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9a857-c38f-4d2e-a6fb-635356883a0d None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:59.339284 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-caa9a857-c38f-4d2e-a6fb-635356883a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:59.339284 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9a857-c38f-4d2e-a6fb-635356883a0d None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:59.354548 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ad596-068d-455e-98f4-dffb2ffb642b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:59.356752 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ad596-068d-455e-98f4-dffb2ffb642b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:59.356935 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ad596-068d-455e-98f4-dffb2ffb642b admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:59.357240 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ad596-068d-455e-98f4-dffb2ffb642b admin admin] PATH_INFO: `/v3/projects/62a9704cc0a3433dab02429784d6ee5d` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:59.358849 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67545063-665b-44bc-9338-82ce1b7037dd admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:59.360230 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ad596-068d-455e-98f4-dffb2ffb642b admin admin] RBAC: Authorizing `identity:delete_project(project_id=62a9704cc0a3433dab02429784d6ee5d)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:59.362716 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ad596-068d-455e-98f4-dffb2ffb642b admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:59.372758 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab107dd2-04e0-4f97-8048-af61078f5c3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:59.373145 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab107dd2-04e0-4f97-8048-af61078f5c3f service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:59.373432 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab107dd2-04e0-4f97-8048-af61078f5c3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:59.373713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab107dd2-04e0-4f97-8048-af61078f5c3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:59.414591 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab107dd2-04e0-4f97-8048-af61078f5c3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:59.417165 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab107dd2-04e0-4f97-8048-af61078f5c3f service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:59.436805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1081/2170] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:51:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:51:59.511172 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab107dd2-04e0-4f97-8048-af61078f5c3f service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:51:59.515351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-ab107dd2-04e0-4f97-8048-af61078f5c3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:59.555268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-632ad596-068d-455e-98f4-dffb2ffb642b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62a9704cc0a3433dab02429784d6ee5d', 'request_id': 'req-632ad596-068d-455e-98f4-dffb2ffb642b'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:59.561626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab107dd2-04e0-4f97-8048-af61078f5c3f service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:51:59.580795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-632ad596-068d-455e-98f4-dffb2ffb642b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62a9704cc0a3433dab02429784d6ee5d', 'request_id': 'req-632ad596-068d-455e-98f4-dffb2ffb642b'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:59.617202 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-632ad596-068d-455e-98f4-dffb2ffb642b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62a9704cc0a3433dab02429784d6ee5d', 'request_id': 'req-632ad596-068d-455e-98f4-dffb2ffb642b'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:59.726439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8951ffa-5035-4ad5-b0ac-768101991902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:51:59.727450 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8951ffa-5035-4ad5-b0ac-768101991902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:59.727839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8951ffa-5035-4ad5-b0ac-768101991902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:59.728303 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8951ffa-5035-4ad5-b0ac-768101991902 admin admin] PATH_INFO: `/v3/projects/246fa1f940e846f8b389b1351c4156f1` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:59.729892 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8951ffa-5035-4ad5-b0ac-768101991902 admin admin] RBAC: Authorizing `identity:delete_project(project_id=246fa1f940e846f8b389b1351c4156f1)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:51:59.734456 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8951ffa-5035-4ad5-b0ac-768101991902 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:51:59.775140 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-632ad596-068d-455e-98f4-dffb2ffb642b admin admin] The token cache is being invalidate because project 62a9704cc0a3433dab02429784d6ee5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:51:59.778753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-632ad596-068d-455e-98f4-dffb2ffb642b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:59.778753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-632ad596-068d-455e-98f4-dffb2ffb642b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:51:59.779301 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1090/2171] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:59 2022] DELETE /identity/v3/projects/62a9704cc0a3433dab02429784d6ee5d => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:51:59.784058 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d437b-de42-4b50-bc8f-c24423c7769d None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:59.784360 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d437b-de42-4b50-bc8f-c24423c7769d None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:59.784651 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d437b-de42-4b50-bc8f-c24423c7769d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:59.851579 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-9d9d437b-de42-4b50-bc8f-c24423c7769d None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:59.862254 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-9d9d437b-de42-4b50-bc8f-c24423c7769d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:51:59.897019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1091/2172] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:51:59.917715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d8a98-5e5d-47b3-be14-09327aee0d16 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:51:59.920818 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d8a98-5e5d-47b3-be14-09327aee0d16 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:51:59.921376 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d8a98-5e5d-47b3-be14-09327aee0d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:51:59.970856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-d02d8a98-5e5d-47b3-be14-09327aee0d16 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:51:59.978099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d02d8a98-5e5d-47b3-be14-09327aee0d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:00.023597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b8951ffa-5035-4ad5-b0ac-768101991902 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '246fa1f940e846f8b389b1351c4156f1', 'request_id': 'req-b8951ffa-5035-4ad5-b0ac-768101991902'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:00.040206 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b8951ffa-5035-4ad5-b0ac-768101991902 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '246fa1f940e846f8b389b1351c4156f1', 'request_id': 'req-b8951ffa-5035-4ad5-b0ac-768101991902'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:00.135256 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b8951ffa-5035-4ad5-b0ac-768101991902 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '246fa1f940e846f8b389b1351c4156f1', 'request_id': 'req-b8951ffa-5035-4ad5-b0ac-768101991902'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:00.156435 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b8951ffa-5035-4ad5-b0ac-768101991902 admin admin] The token cache is being invalidate because project 246fa1f940e846f8b389b1351c4156f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:52:00.157140 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b8951ffa-5035-4ad5-b0ac-768101991902 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:00.158466 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b8951ffa-5035-4ad5-b0ac-768101991902 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:00.163894 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1082/2173] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:51:59 2022] DELETE /identity/v3/projects/246fa1f940e846f8b389b1351c4156f1 => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:00.167637 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8951ffa-5035-4ad5-b0ac-768101991902 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:00.169465 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-b8951ffa-5035-4ad5-b0ac-768101991902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:00.190853 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1092/2174] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:51:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:00.192657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d8a98-5e5d-47b3-be14-09327aee0d16 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:00.201399 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8951ffa-5035-4ad5-b0ac-768101991902 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:00.212914 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d8a98-5e5d-47b3-be14-09327aee0d16 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:00.214518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e030558-394d-47f6-b923-a3a06ccbed07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:00.216854 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e030558-394d-47f6-b923-a3a06ccbed07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:00.217385 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e030558-394d-47f6-b923-a3a06ccbed07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:00.217845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e030558-394d-47f6-b923-a3a06ccbed07 admin admin] PATH_INFO: `/v3/users/7185388f206f4289b94182610dc7274e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:00.220236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e030558-394d-47f6-b923-a3a06ccbed07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7185388f206f4289b94182610dc7274e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:00.223499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e030558-394d-47f6-b923-a3a06ccbed07 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:00.226010 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157be103-8958-43f9-a882-cd4444c6cc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:00.226875 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157be103-8958-43f9-a882-cd4444c6cc9c admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:00.227159 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157be103-8958-43f9-a882-cd4444c6cc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:00.227431 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157be103-8958-43f9-a882-cd4444c6cc9c admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:00.228186 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157be103-8958-43f9-a882-cd4444c6cc9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:52:00.229255 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157be103-8958-43f9-a882-cd4444c6cc9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:00.237818 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157be103-8958-43f9-a882-cd4444c6cc9c admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:00.266680 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1093/2175] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:52:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:52:00.271174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157be103-8958-43f9-a882-cd4444c6cc9c admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:00.274695 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-157be103-8958-43f9-a882-cd4444c6cc9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:00.293269 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157be103-8958-43f9-a882-cd4444c6cc9c admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:00.314106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd64cde-b7e9-48b1-b311-a9b9594f6f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:00.314802 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd64cde-b7e9-48b1-b311-a9b9594f6f9a admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:00.315135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd64cde-b7e9-48b1-b311-a9b9594f6f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:00.315422 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd64cde-b7e9-48b1-b311-a9b9594f6f9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:00.316199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd64cde-b7e9-48b1-b311-a9b9594f6f9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:00.318697 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd64cde-b7e9-48b1-b311-a9b9594f6f9a admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:00.479740 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1094/2176] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:52:00 2022] POST /identity/v3/projects => generated 387 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:52:00.481240 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd64cde-b7e9-48b1-b311-a9b9594f6f9a admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:00.485983 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ccd64cde-b7e9-48b1-b311-a9b9594f6f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:00.493073 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0e030558-394d-47f6-b923-a3a06ccbed07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7185388f206f4289b94182610dc7274e'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:00.525989 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0e030558-394d-47f6-b923-a3a06ccbed07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7185388f206f4289b94182610dc7274e'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:00.545626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:00.601294 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd64cde-b7e9-48b1-b311-a9b9594f6f9a admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:00.609310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca24b5-e255-452d-9173-11972b90b3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:00.610246 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca24b5-e255-452d-9173-11972b90b3b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:00.610533 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca24b5-e255-452d-9173-11972b90b3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:00.610803 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca24b5-e255-452d-9173-11972b90b3b7 admin admin] PATH_INFO: `/v3/projects/7a241839e9674489a93a734a6c44c2e0` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:00.611993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca24b5-e255-452d-9173-11972b90b3b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a241839e9674489a93a734a6c44c2e0)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:00.614128 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaca24b5-e255-452d-9173-11972b90b3b7 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:00.614651 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0e030558-394d-47f6-b923-a3a06ccbed07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7185388f206f4289b94182610dc7274e'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:00.626569 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0e030558-394d-47f6-b923-a3a06ccbed07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7185388f206f4289b94182610dc7274e'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:00.626569 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0e030558-394d-47f6-b923-a3a06ccbed07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7185388f206f4289b94182610dc7274e'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:00.652000 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1083/2177] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:52:00 2022] DELETE /identity/v3/users/7185388f206f4289b94182610dc7274e => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:00.659314 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e030558-394d-47f6-b923-a3a06ccbed07 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:00.659314 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-0e030558-394d-47f6-b923-a3a06ccbed07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:00.745836 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e030558-394d-47f6-b923-a3a06ccbed07 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:00.751626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86503d00-bea1-4d63-a119-69ce082534a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:00.752418 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86503d00-bea1-4d63-a119-69ce082534a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:00.752772 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86503d00-bea1-4d63-a119-69ce082534a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:00.753144 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86503d00-bea1-4d63-a119-69ce082534a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:00.754126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86503d00-bea1-4d63-a119-69ce082534a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:00.756453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86503d00-bea1-4d63-a119-69ce082534a8 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:00.812581 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-eaca24b5-e255-452d-9173-11972b90b3b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a241839e9674489a93a734a6c44c2e0', 'request_id': 'req-eaca24b5-e255-452d-9173-11972b90b3b7'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:00.821005 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-eaca24b5-e255-452d-9173-11972b90b3b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a241839e9674489a93a734a6c44c2e0', 'request_id': 'req-eaca24b5-e255-452d-9173-11972b90b3b7'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:00.832654 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-eaca24b5-e255-452d-9173-11972b90b3b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a241839e9674489a93a734a6c44c2e0', 'request_id': 'req-eaca24b5-e255-452d-9173-11972b90b3b7'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:00.871009 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1084/2178] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:52:00 2022] POST /identity/v3/users => generated 294 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:52:00.872135 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86503d00-bea1-4d63-a119-69ce082534a8 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:00.895209 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86503d00-bea1-4d63-a119-69ce082534a8 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:00.921732 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-eaca24b5-e255-452d-9173-11972b90b3b7 admin admin] The token cache is being invalidate because project 7a241839e9674489a93a734a6c44c2e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:52:00.922170 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-eaca24b5-e255-452d-9173-11972b90b3b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:00.923021 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-eaca24b5-e255-452d-9173-11972b90b3b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:00.928121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1095/2179] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:52:00 2022] DELETE /identity/v3/projects/7a241839e9674489a93a734a6c44c2e0 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:00.930910 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a55d2-6684-4afc-b80b-b5fab6ecfb47 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:00.931205 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a55d2-6684-4afc-b80b-b5fab6ecfb47 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:00.931484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47a55d2-6684-4afc-b80b-b5fab6ecfb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:00.999950 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-a47a55d2-6684-4afc-b80b-b5fab6ecfb47 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:01.006108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-a47a55d2-6684-4afc-b80b-b5fab6ecfb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:01.009119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d2b54-f776-48fc-889e-e86b436b0133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:01.009978 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d2b54-f776-48fc-889e-e86b436b0133 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:01.010444 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d2b54-f776-48fc-889e-e86b436b0133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:01.010828 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d2b54-f776-48fc-889e-e86b436b0133 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:01.011773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d2b54-f776-48fc-889e-e86b436b0133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:01.013865 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d2b54-f776-48fc-889e-e86b436b0133 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:01.050325 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1096/2180] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:01.051921 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1085/2181] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:00 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:01.061308 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a55d2-6684-4afc-b80b-b5fab6ecfb47 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:01.078179 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f28c8-f219-48ae-8727-c79ae59780b9 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:01.078526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f28c8-f219-48ae-8727-c79ae59780b9 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:01.078766 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f28c8-f219-48ae-8727-c79ae59780b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:01.086809 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-a47a55d2-6684-4afc-b80b-b5fab6ecfb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:01.118073 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47a55d2-6684-4afc-b80b-b5fab6ecfb47 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:01.135377 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-987f28c8-f219-48ae-8727-c79ae59780b9 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:01.136405 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45048ea6-ac49-4dd4-aed0-73c25950d8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:01.137135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45048ea6-ac49-4dd4-aed0-73c25950d8ae admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:01.137449 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45048ea6-ac49-4dd4-aed0-73c25950d8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:01.137758 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45048ea6-ac49-4dd4-aed0-73c25950d8ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:01.138789 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45048ea6-ac49-4dd4-aed0-73c25950d8ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:01.140682 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-987f28c8-f219-48ae-8727-c79ae59780b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:01.143255 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45048ea6-ac49-4dd4-aed0-73c25950d8ae admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:01.172022 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1097/2182] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:01 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:01.172269 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45048ea6-ac49-4dd4-aed0-73c25950d8ae admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:01.178173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1086/2183] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:01.184448 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f28c8-f219-48ae-8727-c79ae59780b9 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:01.200598 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987f28c8-f219-48ae-8727-c79ae59780b9 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:01.203612 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45048ea6-ac49-4dd4-aed0-73c25950d8ae admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:01.207178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a0697-9e88-49b2-afb2-25aa48a2c834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:01.207962 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a0697-9e88-49b2-afb2-25aa48a2c834 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:01.208273 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a0697-9e88-49b2-afb2-25aa48a2c834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:01.208559 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808a0697-9e88-49b2-afb2-25aa48a2c834 admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:01.209094 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73c839-3f34-42be-b0ab-a14c2e35cd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:01.228200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a0697-9e88-49b2-afb2-25aa48a2c834 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:52:01.228200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a0697-9e88-49b2-afb2-25aa48a2c834 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:01.228454 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73c839-3f34-42be-b0ab-a14c2e35cd82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:01.228454 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73c839-3f34-42be-b0ab-a14c2e35cd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:01.228454 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc73c839-3f34-42be-b0ab-a14c2e35cd82 admin admin] PATH_INFO: `/v3/projects/7423a054a3af4b7d8e16f5050a5ea052/users/9304c2fa316b4196a56fb2e5d10127bd/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:01.228667 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808a0697-9e88-49b2-afb2-25aa48a2c834 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:01.229480 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1087/2184] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:52:01.230432 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a0697-9e88-49b2-afb2-25aa48a2c834 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:01.237216 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc73c839-3f34-42be-b0ab-a14c2e35cd82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7423a054a3af4b7d8e16f5050a5ea052, user_id=9304c2fa316b4196a56fb2e5d10127bd, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:01.240258 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc73c839-3f34-42be-b0ab-a14c2e35cd82 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:01.242271 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808a0697-9e88-49b2-afb2-25aa48a2c834 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:01.247751 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2a26c-2907-46d0-83a6-f7c58e5701b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:01.248413 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2a26c-2907-46d0-83a6-f7c58e5701b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:01.248701 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2a26c-2907-46d0-83a6-f7c58e5701b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:01.249010 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c2a26c-2907-46d0-83a6-f7c58e5701b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:01.249746 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c2a26c-2907-46d0-83a6-f7c58e5701b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:01.251863 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c2a26c-2907-46d0-83a6-f7c58e5701b7 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:01.700657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:01.710527 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1088/2185] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:52:01 2022] POST /identity/v3/projects => generated 377 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:52:01.711527 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2a26c-2907-46d0-83a6-f7c58e5701b7 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:01.713441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1098/2186] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:52:01 2022] PUT /identity/v3/projects/7423a054a3af4b7d8e16f5050a5ea052/users/9304c2fa316b4196a56fb2e5d10127bd/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:01.714445 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73c839-3f34-42be-b0ab-a14c2e35cd82 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:01.727349 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-97c2a26c-2907-46d0-83a6-f7c58e5701b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:01.838957 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c2a26c-2907-46d0-83a6-f7c58e5701b7 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:01.846041 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f84c90-c16a-4b2c-97f8-505e810596fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:01.848118 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f84c90-c16a-4b2c-97f8-505e810596fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:01.848566 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f84c90-c16a-4b2c-97f8-505e810596fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:01.849163 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f84c90-c16a-4b2c-97f8-505e810596fc admin admin] PATH_INFO: `/v3/projects/91988bc3a29d4d92a1f5f9576512b62d` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:01.851216 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f84c90-c16a-4b2c-97f8-505e810596fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=91988bc3a29d4d92a1f5f9576512b62d)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:01.853516 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f84c90-c16a-4b2c-97f8-505e810596fc admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:01.882540 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc73c839-3f34-42be-b0ab-a14c2e35cd82 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:01.894565 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbc45f-083a-4e26-9034-40d22f95bc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:01.895266 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbc45f-083a-4e26-9034-40d22f95bc1f admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:01.897985 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbc45f-083a-4e26-9034-40d22f95bc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:01.897985 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbc45f-083a-4e26-9034-40d22f95bc1f admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:01.897985 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbc45f-083a-4e26-9034-40d22f95bc1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:01.899098 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bbc45f-083a-4e26-9034-40d22f95bc1f admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:02.023456 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-59f84c90-c16a-4b2c-97f8-505e810596fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91988bc3a29d4d92a1f5f9576512b62d', 'request_id': 'req-59f84c90-c16a-4b2c-97f8-505e810596fc'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:02.029393 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-59f84c90-c16a-4b2c-97f8-505e810596fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91988bc3a29d4d92a1f5f9576512b62d', 'request_id': 'req-59f84c90-c16a-4b2c-97f8-505e810596fc'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:02.030098 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1099/2187] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:52:01 2022] POST /identity/v3/users => generated 289 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:52:02.031665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae737be-a02f-477d-8a81-dd3962c7eb51 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:02.033615 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae737be-a02f-477d-8a81-dd3962c7eb51 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:02.034587 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae737be-a02f-477d-8a81-dd3962c7eb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:02.082386 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-cae737be-a02f-477d-8a81-dd3962c7eb51 None None] MFA Rules not processed for user `9304c2fa316b4196a56fb2e5d10127bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:02.086062 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-cae737be-a02f-477d-8a81-dd3962c7eb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:02.151112 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-59f84c90-c16a-4b2c-97f8-505e810596fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91988bc3a29d4d92a1f5f9576512b62d', 'request_id': 'req-59f84c90-c16a-4b2c-97f8-505e810596fc'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:02.170487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-59f84c90-c16a-4b2c-97f8-505e810596fc admin admin] The token cache is being invalidate because project 91988bc3a29d4d92a1f5f9576512b62d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:52:02.177442 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-59f84c90-c16a-4b2c-97f8-505e810596fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:02.179017 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-59f84c90-c16a-4b2c-97f8-505e810596fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:02.184350 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1100/2188] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:02.184350 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae737be-a02f-477d-8a81-dd3962c7eb51 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:02.190559 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1089/2189] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:52:01 2022] DELETE /identity/v3/projects/91988bc3a29d4d92a1f5f9576512b62d => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:02.193967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f84c90-c16a-4b2c-97f8-505e810596fc admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:02.198635 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-59f84c90-c16a-4b2c-97f8-505e810596fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:02.273174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae737be-a02f-477d-8a81-dd3962c7eb51 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:02.279057 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f84c90-c16a-4b2c-97f8-505e810596fc admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:02.280927 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e76c09-3c7f-488e-b20a-005c8a4303c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:02.281829 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e76c09-3c7f-488e-b20a-005c8a4303c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:02.283038 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e76c09-3c7f-488e-b20a-005c8a4303c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:02.283334 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e76c09-3c7f-488e-b20a-005c8a4303c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:02.284079 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e76c09-3c7f-488e-b20a-005c8a4303c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:02.284985 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2332f4a-fe45-46d0-aab4-d9623947d0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:02.285720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2332f4a-fe45-46d0-aab4-d9623947d0b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:02.288615 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2332f4a-fe45-46d0-aab4-d9623947d0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:02.288742 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e76c09-3c7f-488e-b20a-005c8a4303c5 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:02.288890 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2332f4a-fe45-46d0-aab4-d9623947d0b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:02.288890 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2332f4a-fe45-46d0-aab4-d9623947d0b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:02.290962 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2332f4a-fe45-46d0-aab4-d9623947d0b7 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:02.301126 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1101/2190] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:02.301126 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145ec4d-2bf7-4786-acf6-b31bceb64834 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:02.301126 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145ec4d-2bf7-4786-acf6-b31bceb64834 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:02.301126 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145ec4d-2bf7-4786-acf6-b31bceb64834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:02.326123 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-2145ec4d-2bf7-4786-acf6-b31bceb64834 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:02.327496 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-2145ec4d-2bf7-4786-acf6-b31bceb64834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:02.366886 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1102/2191] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:02.366886 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145ec4d-2bf7-4786-acf6-b31bceb64834 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:02.368714 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-2145ec4d-2bf7-4786-acf6-b31bceb64834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:02.383631 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1090/2192] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:52:02 2022] POST /identity/v3/projects => generated 385 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:52:02.385608 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561328e-0982-4395-a896-7d57e3b7ac1b None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:02.386129 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561328e-0982-4395-a896-7d57e3b7ac1b None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:02.386626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561328e-0982-4395-a896-7d57e3b7ac1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:02.394803 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145ec4d-2bf7-4786-acf6-b31bceb64834 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:02.421052 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-6561328e-0982-4395-a896-7d57e3b7ac1b None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:02.422577 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-6561328e-0982-4395-a896-7d57e3b7ac1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:02.472405 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b01287-2958-438a-9707-b69bbd4a962a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:02.473289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b01287-2958-438a-9707-b69bbd4a962a admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:02.473638 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b01287-2958-438a-9707-b69bbd4a962a admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:02.473987 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b01287-2958-438a-9707-b69bbd4a962a admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:02.474858 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b01287-2958-438a-9707-b69bbd4a962a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:02.477590 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b01287-2958-438a-9707-b69bbd4a962a admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:02.495692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1103/2193] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:02.495692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b01287-2958-438a-9707-b69bbd4a962a admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:02.498475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1091/2194] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:02.499660 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561328e-0982-4395-a896-7d57e3b7ac1b None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:02.511604 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b01287-2958-438a-9707-b69bbd4a962a admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:02.517263 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce26079d-ec00-4d27-98c8-cf67a3372e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:02.517627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561328e-0982-4395-a896-7d57e3b7ac1b None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:02.517954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26079d-ec00-4d27-98c8-cf67a3372e3b admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:02.518238 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26079d-ec00-4d27-98c8-cf67a3372e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:02.518515 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce26079d-ec00-4d27-98c8-cf67a3372e3b admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:02.519227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce26079d-ec00-4d27-98c8-cf67a3372e3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:02.521278 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce26079d-ec00-4d27-98c8-cf67a3372e3b admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:02.525767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d1a60-0819-438e-8671-c1af5bda9b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:02.530953 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d1a60-0819-438e-8671-c1af5bda9b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:02.531452 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d1a60-0819-438e-8671-c1af5bda9b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:02.532044 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d1a60-0819-438e-8671-c1af5bda9b43 admin admin] PATH_INFO: `/v3/projects/ad4a9053ff334183a580cd1dbf8eea18/users/e279760852fd4370b372a09c8148bb8b/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:02.566594 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d1a60-0819-438e-8671-c1af5bda9b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad4a9053ff334183a580cd1dbf8eea18, user_id=e279760852fd4370b372a09c8148bb8b, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:02.570847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42d1a60-0819-438e-8671-c1af5bda9b43 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:02.652784 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1104/2195] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:52:02 2022] POST /identity/v3/users => generated 299 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:52:02.656877 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce26079d-ec00-4d27-98c8-cf67a3372e3b admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:02.680280 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1092/2196] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:52:02 2022] PUT /identity/v3/projects/ad4a9053ff334183a580cd1dbf8eea18/users/e279760852fd4370b372a09c8148bb8b/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:02.680280 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d1a60-0819-438e-8671-c1af5bda9b43 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:02.683522 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce26079d-ec00-4d27-98c8-cf67a3372e3b admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:02.796527 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bdce37-5273-442c-9f30-d7e50ced922c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:02.806505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdce37-5273-442c-9f30-d7e50ced922c admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:02.807102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdce37-5273-442c-9f30-d7e50ced922c admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:02.808167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bdce37-5273-442c-9f30-d7e50ced922c admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:02.809451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bdce37-5273-442c-9f30-d7e50ced922c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:52:02.811279 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bdce37-5273-442c-9f30-d7e50ced922c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:02.813197 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bdce37-5273-442c-9f30-d7e50ced922c admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:02.814590 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d1a60-0819-438e-8671-c1af5bda9b43 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:02.823875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262addb-8267-426f-b5cd-c46e6494147e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:02.827802 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262addb-8267-426f-b5cd-c46e6494147e admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:02.828097 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262addb-8267-426f-b5cd-c46e6494147e admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:02.828381 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1262addb-8267-426f-b5cd-c46e6494147e admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:02.834271 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262addb-8267-426f-b5cd-c46e6494147e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:02.840459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1105/2197] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:52:02.843460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bb01a-c1e3-4497-a7ce-14ffb3a178ed None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:02.843899 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1262addb-8267-426f-b5cd-c46e6494147e admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:02.845586 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bb01a-c1e3-4497-a7ce-14ffb3a178ed None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:02.846011 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6bb01a-c1e3-4497-a7ce-14ffb3a178ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:02.864121 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1093/2198] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:02 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:02.877274 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262addb-8267-426f-b5cd-c46e6494147e admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:02.913281 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1262addb-8267-426f-b5cd-c46e6494147e admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:02.920012 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a7036c-d8d2-4c37-837f-d8f476217881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:02.920700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7036c-d8d2-4c37-837f-d8f476217881 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:02.924544 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7036c-d8d2-4c37-837f-d8f476217881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:02.924544 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a7036c-d8d2-4c37-837f-d8f476217881 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:02.924544 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a7036c-d8d2-4c37-837f-d8f476217881 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:02.924544 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a7036c-d8d2-4c37-837f-d8f476217881 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:02.938304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-ae6bb01a-c1e3-4497-a7ce-14ffb3a178ed None None] MFA Rules not processed for user `e279760852fd4370b372a09c8148bb8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:02.939558 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ae6bb01a-c1e3-4497-a7ce-14ffb3a178ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:03.065095 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1106/2199] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:02 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:03.065095 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6bb01a-c1e3-4497-a7ce-14ffb3a178ed None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:03.065367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1094/2200] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:52:02 2022] POST /identity/v3/projects => generated 371 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:52:03.065367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a7036c-d8d2-4c37-837f-d8f476217881 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:03.079839 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6bb01a-c1e3-4497-a7ce-14ffb3a178ed None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:03.159427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b60cf-050c-463c-b17b-edf7d3e12a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:03.160883 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b60cf-050c-463c-b17b-edf7d3e12a82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:03.161404 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b60cf-050c-463c-b17b-edf7d3e12a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:03.162006 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b60cf-050c-463c-b17b-edf7d3e12a82 admin admin] PATH_INFO: `/v3/projects/420dccccc18c49cb89708a430be1c82b/users/fc53af1f5e5c47ff8ca212f5a1d1ef43/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:03.183971 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b60cf-050c-463c-b17b-edf7d3e12a82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=420dccccc18c49cb89708a430be1c82b, user_id=fc53af1f5e5c47ff8ca212f5a1d1ef43, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:03.187852 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b60cf-050c-463c-b17b-edf7d3e12a82 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:03.219978 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a7036c-d8d2-4c37-837f-d8f476217881 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:03.227661 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07376c18-1a9d-421d-808b-9a1b059f0cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:03.228700 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07376c18-1a9d-421d-808b-9a1b059f0cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:03.229235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07376c18-1a9d-421d-808b-9a1b059f0cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:03.229895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07376c18-1a9d-421d-808b-9a1b059f0cf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:03.230863 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07376c18-1a9d-421d-808b-9a1b059f0cf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:03.233621 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07376c18-1a9d-421d-808b-9a1b059f0cf7 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:03.282116 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1107/2201] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:52:03 2022] PUT /identity/v3/projects/420dccccc18c49cb89708a430be1c82b/users/fc53af1f5e5c47ff8ca212f5a1d1ef43/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:03.283001 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b60cf-050c-463c-b17b-edf7d3e12a82 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:03.370417 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b60cf-050c-463c-b17b-edf7d3e12a82 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:03.375476 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1856a9c-b9d8-47ce-864f-926f87fc6daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:03.376456 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1856a9c-b9d8-47ce-864f-926f87fc6daf admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:03.376817 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1856a9c-b9d8-47ce-864f-926f87fc6daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:03.377241 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1856a9c-b9d8-47ce-864f-926f87fc6daf admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:03.378115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1856a9c-b9d8-47ce-864f-926f87fc6daf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:03.380800 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1856a9c-b9d8-47ce-864f-926f87fc6daf admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:03.403283 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1095/2202] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:52:03 2022] POST /identity/v3/projects => generated 377 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:52:03.404815 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07376c18-1a9d-421d-808b-9a1b059f0cf7 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:03.474667 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07376c18-1a9d-421d-808b-9a1b059f0cf7 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:03.480183 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd188f0-1801-41c7-a940-62c7ea1bd860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:03.481115 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd188f0-1801-41c7-a940-62c7ea1bd860 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:03.481371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd188f0-1801-41c7-a940-62c7ea1bd860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:03.482623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd188f0-1801-41c7-a940-62c7ea1bd860 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:03.482623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd188f0-1801-41c7-a940-62c7ea1bd860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:03.484198 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd188f0-1801-41c7-a940-62c7ea1bd860 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:03.502011 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1108/2203] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:52:03 2022] POST /identity/v3/users => generated 286 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:52:03.502011 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1856a9c-b9d8-47ce-864f-926f87fc6daf admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:03.507037 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1096/2204] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:03.507037 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd188f0-1801-41c7-a940-62c7ea1bd860 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:03.521377 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1856a9c-b9d8-47ce-864f-926f87fc6daf admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:03.529726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd188f0-1801-41c7-a940-62c7ea1bd860 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:03.539381 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5afe03e-0a8d-43a3-8e5a-a7cff7cbda7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:03.540073 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afe03e-0a8d-43a3-8e5a-a7cff7cbda7e admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:03.540367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afe03e-0a8d-43a3-8e5a-a7cff7cbda7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:03.540656 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afe03e-0a8d-43a3-8e5a-a7cff7cbda7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:03.541441 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5afe03e-0a8d-43a3-8e5a-a7cff7cbda7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:03.543075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5afe03e-0a8d-43a3-8e5a-a7cff7cbda7e admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:03.543875 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47ef530-00cf-4419-9950-22a5c1f96092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:03.544704 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ef530-00cf-4419-9950-22a5c1f96092 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:03.546184 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ef530-00cf-4419-9950-22a5c1f96092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:03.546575 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47ef530-00cf-4419-9950-22a5c1f96092 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:03.548328 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ef530-00cf-4419-9950-22a5c1f96092 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:03.551658 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47ef530-00cf-4419-9950-22a5c1f96092 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:03.554446 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1097/2205] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:03.555390 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5afe03e-0a8d-43a3-8e5a-a7cff7cbda7e admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:03.573001 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5afe03e-0a8d-43a3-8e5a-a7cff7cbda7e admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:03.578937 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0daa5-e0fa-41d7-a01b-227af504edb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:03.579920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0daa5-e0fa-41d7-a01b-227af504edb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:03.580217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0daa5-e0fa-41d7-a01b-227af504edb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:03.580563 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0daa5-e0fa-41d7-a01b-227af504edb6 admin admin] PATH_INFO: `/v3/projects/420dccccc18c49cb89708a430be1c82b/users/fc53af1f5e5c47ff8ca212f5a1d1ef43/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:03.588594 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0daa5-e0fa-41d7-a01b-227af504edb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=420dccccc18c49cb89708a430be1c82b, user_id=fc53af1f5e5c47ff8ca212f5a1d1ef43, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:03.591943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0daa5-e0fa-41d7-a01b-227af504edb6 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:03.610080 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1098/2206] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:52:03 2022] PUT /identity/v3/projects/420dccccc18c49cb89708a430be1c82b/users/fc53af1f5e5c47ff8ca212f5a1d1ef43/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:03.617951 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0daa5-e0fa-41d7-a01b-227af504edb6 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:03.656933 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1109/2207] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:52:03 2022] POST /identity/v3/users => generated 289 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:52:03.658880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828cc58-72bb-49d7-bebf-4bbf29e49fea None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:03.660250 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828cc58-72bb-49d7-bebf-4bbf29e49fea None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:03.661921 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828cc58-72bb-49d7-bebf-4bbf29e49fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:03.703369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:03.710728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-2828cc58-72bb-49d7-bebf-4bbf29e49fea None None] MFA Rules not processed for user `fc53af1f5e5c47ff8ca212f5a1d1ef43`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:03.715056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-2828cc58-72bb-49d7-bebf-4bbf29e49fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:03.747182 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0daa5-e0fa-41d7-a01b-227af504edb6 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:03.752661 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e12b5c-7c17-4725-a21f-b63d59b12bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:03.753391 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e12b5c-7c17-4725-a21f-b63d59b12bbd admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:03.753682 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e12b5c-7c17-4725-a21f-b63d59b12bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:03.753963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e12b5c-7c17-4725-a21f-b63d59b12bbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:03.754746 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e12b5c-7c17-4725-a21f-b63d59b12bbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:03.756419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e12b5c-7c17-4725-a21f-b63d59b12bbd admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:03.771270 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1099/2208] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:03.772923 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e12b5c-7c17-4725-a21f-b63d59b12bbd admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:03.787182 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e12b5c-7c17-4725-a21f-b63d59b12bbd admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:03.793157 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e76a7c-edd8-4b4b-ac08-fdc432b0c528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:03.793847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e76a7c-edd8-4b4b-ac08-fdc432b0c528 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:03.794134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e76a7c-edd8-4b4b-ac08-fdc432b0c528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:03.794416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e76a7c-edd8-4b4b-ac08-fdc432b0c528 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:03.795161 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e76a7c-edd8-4b4b-ac08-fdc432b0c528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:03.796825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e76a7c-edd8-4b4b-ac08-fdc432b0c528 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:03.820998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1100/2209] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:03.820998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e76a7c-edd8-4b4b-ac08-fdc432b0c528 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:03.835621 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e76a7c-edd8-4b4b-ac08-fdc432b0c528 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:03.841078 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07869f51-f37b-4308-888c-0e9299c22687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:03.842059 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07869f51-f37b-4308-888c-0e9299c22687 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:03.842303 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07869f51-f37b-4308-888c-0e9299c22687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:03.842584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07869f51-f37b-4308-888c-0e9299c22687 admin admin] PATH_INFO: `/v3/projects/729158c0db83438a9b762aae3f0009a2/users/83d3494760b84bc1aad951bba11caea3/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:03.848128 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1110/2210] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:03 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:03.850172 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828cc58-72bb-49d7-bebf-4bbf29e49fea None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:03.861963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07869f51-f37b-4308-888c-0e9299c22687 admin admin] RBAC: Authorizing `identity:create_grant(project_id=729158c0db83438a9b762aae3f0009a2, user_id=83d3494760b84bc1aad951bba11caea3, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:03.865750 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07869f51-f37b-4308-888c-0e9299c22687 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:03.874879 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828cc58-72bb-49d7-bebf-4bbf29e49fea None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:03.880090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8f622-dc5a-4725-883b-0aad0055fdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:03.880837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8f622-dc5a-4725-883b-0aad0055fdf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:03.881174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8f622-dc5a-4725-883b-0aad0055fdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:03.881468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a8f622-dc5a-4725-883b-0aad0055fdf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:03.882601 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8f622-dc5a-4725-883b-0aad0055fdf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:03.884999 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a8f622-dc5a-4725-883b-0aad0055fdf0 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:03.908437 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1111/2211] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:03 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:03.910024 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8f622-dc5a-4725-883b-0aad0055fdf0 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:03.914113 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-c7a8f622-dc5a-4725-883b-0aad0055fdf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:03.950626 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1101/2212] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:52:03 2022] PUT /identity/v3/projects/729158c0db83438a9b762aae3f0009a2/users/83d3494760b84bc1aad951bba11caea3/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:03.957047 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07869f51-f37b-4308-888c-0e9299c22687 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:04.015342 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a8f622-dc5a-4725-883b-0aad0055fdf0 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:04.069769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07869f51-f37b-4308-888c-0e9299c22687 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:04.075534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01124821-9249-403b-bc8a-f54eecc0d9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:04.076540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01124821-9249-403b-bc8a-f54eecc0d9db admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:04.077066 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01124821-9249-403b-bc8a-f54eecc0d9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:04.077962 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01124821-9249-403b-bc8a-f54eecc0d9db admin admin] PATH_INFO: `/v3/projects/56ecbb75334f465c97d12918f529283b/users/9c31944aa13e4ee186dc6594ed198ad6/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:04.096457 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255094eb-4350-4f01-a130-86885fb7eaed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:04.099252 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255094eb-4350-4f01-a130-86885fb7eaed service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:04.099252 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255094eb-4350-4f01-a130-86885fb7eaed service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:04.099252 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255094eb-4350-4f01-a130-86885fb7eaed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:04.113692 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01124821-9249-403b-bc8a-f54eecc0d9db admin admin] RBAC: Authorizing `identity:create_grant(project_id=56ecbb75334f465c97d12918f529283b, user_id=9c31944aa13e4ee186dc6594ed198ad6, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:04.113933 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=81,keystone:DELETE=15,keystone:INSERT=4 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:04.114112 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01124821-9249-403b-bc8a-f54eecc0d9db admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:04.212995 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1102/2213] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:52:03 2022] PUT /identity/v3/projects/56ecbb75334f465c97d12918f529283b/users/9c31944aa13e4ee186dc6594ed198ad6/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:04.212995 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a0c5b-812f-4ada-8c07-608279ed7fff None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:04.212995 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a0c5b-812f-4ada-8c07-608279ed7fff None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:04.212995 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a0c5b-812f-4ada-8c07-608279ed7fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:04.214784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:04.218370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255094eb-4350-4f01-a130-86885fb7eaed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:04.221946 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255094eb-4350-4f01-a130-86885fb7eaed service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:04.227141 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-690a0c5b-812f-4ada-8c07-608279ed7fff None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:04.228351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-690a0c5b-812f-4ada-8c07-608279ed7fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:04.232473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1112/2214] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:52:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:52:04.233963 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4160a4-8d95-4689-bfbe-e7e6335cd1b8 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:04.234260 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4160a4-8d95-4689-bfbe-e7e6335cd1b8 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:04.234544 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4160a4-8d95-4689-bfbe-e7e6335cd1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:04.282263 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-4d4160a4-8d95-4689-bfbe-e7e6335cd1b8 None None] MFA Rules not processed for user `9c31944aa13e4ee186dc6594ed198ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:04.283549 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-4d4160a4-8d95-4689-bfbe-e7e6335cd1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:04.312286 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1103/2215] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:04.330587 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a0c5b-812f-4ada-8c07-608279ed7fff None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:04.344518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a0c5b-812f-4ada-8c07-608279ed7fff None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:04.354005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f71ce-fc14-4d71-bf1b-2528346c2f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:04.357023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f71ce-fc14-4d71-bf1b-2528346c2f1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:04.357359 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f71ce-fc14-4d71-bf1b-2528346c2f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:04.357616 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f71ce-fc14-4d71-bf1b-2528346c2f1b admin admin] PATH_INFO: `/v3/users/9304c2fa316b4196a56fb2e5d10127bd` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:04.360632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4f71ce-fc14-4d71-bf1b-2528346c2f1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9304c2fa316b4196a56fb2e5d10127bd)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:04.366751 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4f71ce-fc14-4d71-bf1b-2528346c2f1b admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:04.410910 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1113/2216] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:04.411968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4160a4-8d95-4689-bfbe-e7e6335cd1b8 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:04.418902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-4d4160a4-8d95-4689-bfbe-e7e6335cd1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:04.504470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4160a4-8d95-4689-bfbe-e7e6335cd1b8 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:04.509867 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d114a7fd-19f4-4032-af62-ad5608f5908d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:04.510559 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114a7fd-19f4-4032-af62-ad5608f5908d service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:04.510834 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114a7fd-19f4-4032-af62-ad5608f5908d service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:04.511111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114a7fd-19f4-4032-af62-ad5608f5908d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:04.529819 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d114a7fd-19f4-4032-af62-ad5608f5908d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:04.531422 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d114a7fd-19f4-4032-af62-ad5608f5908d service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:04.563385 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1114/2217] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:52:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:52:04.564539 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-8d4f71ce-fc14-4d71-bf1b-2528346c2f1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9304c2fa316b4196a56fb2e5d10127bd'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:04.565819 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d114a7fd-19f4-4032-af62-ad5608f5908d service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:04.588395 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d114a7fd-19f4-4032-af62-ad5608f5908d service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:04.599080 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-8d4f71ce-fc14-4d71-bf1b-2528346c2f1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9304c2fa316b4196a56fb2e5d10127bd'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:04.602299 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e9ae3-e884-4e2d-b5e7-a1589f039bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:04.603066 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e9ae3-e884-4e2d-b5e7-a1589f039bcc admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:04.603350 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e9ae3-e884-4e2d-b5e7-a1589f039bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:04.603725 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02e9ae3-e884-4e2d-b5e7-a1589f039bcc admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:04.604907 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e9ae3-e884-4e2d-b5e7-a1589f039bcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:04.607429 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02e9ae3-e884-4e2d-b5e7-a1589f039bcc admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:04.661819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-8d4f71ce-fc14-4d71-bf1b-2528346c2f1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9304c2fa316b4196a56fb2e5d10127bd'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:04.663705 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-8d4f71ce-fc14-4d71-bf1b-2528346c2f1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9304c2fa316b4196a56fb2e5d10127bd'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:04.665151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-8d4f71ce-fc14-4d71-bf1b-2528346c2f1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9304c2fa316b4196a56fb2e5d10127bd'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:04.718880 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1104/2218] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:52:04 2022] DELETE /identity/v3/users/9304c2fa316b4196a56fb2e5d10127bd => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:04.728341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1115/2219] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:52:04 2022] POST /identity/v3/projects => generated 375 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:52:04.728804 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7ab08-6ffc-4cff-a0b1-0ec1a6bfee1c None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:04.729341 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7ab08-6ffc-4cff-a0b1-0ec1a6bfee1c None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:04.729705 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7ab08-6ffc-4cff-a0b1-0ec1a6bfee1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:04.729817 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e9ae3-e884-4e2d-b5e7-a1589f039bcc admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:04.731506 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-e02e9ae3-e884-4e2d-b5e7-a1589f039bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:04.774257 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-22b7ab08-6ffc-4cff-a0b1-0ec1a6bfee1c None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:04.778018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-22b7ab08-6ffc-4cff-a0b1-0ec1a6bfee1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:04.876603 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02e9ae3-e884-4e2d-b5e7-a1589f039bcc admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:04.893526 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc8c77-14d6-4388-b3bd-92a7b3218370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:04.895253 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc8c77-14d6-4388-b3bd-92a7b3218370 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:04.895810 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc8c77-14d6-4388-b3bd-92a7b3218370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:04.898359 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc8c77-14d6-4388-b3bd-92a7b3218370 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:04.899398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc8c77-14d6-4388-b3bd-92a7b3218370 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:04.905398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc8c77-14d6-4388-b3bd-92a7b3218370 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:04.922586 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1105/2220] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:04.935064 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7ab08-6ffc-4cff-a0b1-0ec1a6bfee1c None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:04.937415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-22b7ab08-6ffc-4cff-a0b1-0ec1a6bfee1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:05.032718 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7ab08-6ffc-4cff-a0b1-0ec1a6bfee1c None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:05.039226 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1116/2221] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:52:04 2022] POST /identity/v3/users => generated 294 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:52:05.044852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d24074-d195-439c-85d6-c108e75a83d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:05.045045 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc8c77-14d6-4388-b3bd-92a7b3218370 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:05.045499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d24074-d195-439c-85d6-c108e75a83d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:05.046117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d24074-d195-439c-85d6-c108e75a83d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:05.047346 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d24074-d195-439c-85d6-c108e75a83d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:05.060919 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc8c77-14d6-4388-b3bd-92a7b3218370 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:05.066977 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54034d3-3b3f-4f18-a2ad-3d67fa7f41f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:05.067667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54034d3-3b3f-4f18-a2ad-3d67fa7f41f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:05.067958 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54034d3-3b3f-4f18-a2ad-3d67fa7f41f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:05.068243 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54034d3-3b3f-4f18-a2ad-3d67fa7f41f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:05.069113 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54034d3-3b3f-4f18-a2ad-3d67fa7f41f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:05.071476 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54034d3-3b3f-4f18-a2ad-3d67fa7f41f3 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:05.087720 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1117/2222] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:05.088453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d24074-d195-439c-85d6-c108e75a83d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:05.089284 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54034d3-3b3f-4f18-a2ad-3d67fa7f41f3 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:05.092926 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d24074-d195-439c-85d6-c108e75a83d4 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:05.103677 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54034d3-3b3f-4f18-a2ad-3d67fa7f41f3 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:05.108875 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb958b0-e035-49a3-9934-309dfe2b091b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:05.109354 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb958b0-e035-49a3-9934-309dfe2b091b admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:05.109704 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb958b0-e035-49a3-9934-309dfe2b091b admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:05.110195 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb958b0-e035-49a3-9934-309dfe2b091b admin admin] PATH_INFO: `/v3/projects/973f04b8ad984f66b8825e408abc005c/users/5816a1004aa44dea955938edab17ab99/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:05.133307 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1106/2223] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:52:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:52:05.144889 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb958b0-e035-49a3-9934-309dfe2b091b admin admin] RBAC: Authorizing `identity:create_grant(project_id=973f04b8ad984f66b8825e408abc005c, user_id=5816a1004aa44dea955938edab17ab99, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:05.154760 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb958b0-e035-49a3-9934-309dfe2b091b admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:05.243581 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d24074-d195-439c-85d6-c108e75a83d4 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:05.254275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1118/2224] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:52:05 2022] PUT /identity/v3/projects/973f04b8ad984f66b8825e408abc005c/users/5816a1004aa44dea955938edab17ab99/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:52:05.276436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d24074-d195-439c-85d6-c108e75a83d4 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:05.378122 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff0470-71b1-4f9f-b426-ca5c5a8e5028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:05.378953 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff0470-71b1-4f9f-b426-ca5c5a8e5028 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:05.379468 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff0470-71b1-4f9f-b426-ca5c5a8e5028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:05.379810 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff0470-71b1-4f9f-b426-ca5c5a8e5028 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:05.383093 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff0470-71b1-4f9f-b426-ca5c5a8e5028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:05.386759 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff0470-71b1-4f9f-b426-ca5c5a8e5028 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:05.423293 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1107/2225] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:05 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:05.431704 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb958b0-e035-49a3-9934-309dfe2b091b admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:05.458477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb958b0-e035-49a3-9934-309dfe2b091b admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:05.488307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7be25-b3f8-48aa-91a6-dbd12ab8d7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:05.489394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7be25-b3f8-48aa-91a6-dbd12ab8d7e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:05.489845 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7be25-b3f8-48aa-91a6-dbd12ab8d7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:05.490226 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7be25-b3f8-48aa-91a6-dbd12ab8d7e7 admin admin] PATH_INFO: `/v3/projects/973f04b8ad984f66b8825e408abc005c/users/5816a1004aa44dea955938edab17ab99/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:05.493710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7be25-b3f8-48aa-91a6-dbd12ab8d7e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=973f04b8ad984f66b8825e408abc005c, user_id=5816a1004aa44dea955938edab17ab99, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:05.498411 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7be25-b3f8-48aa-91a6-dbd12ab8d7e7 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:05.536720 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1119/2226] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:52:05 2022] PUT /identity/v3/projects/973f04b8ad984f66b8825e408abc005c/users/5816a1004aa44dea955938edab17ab99/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:05.540185 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44407386-7a6d-45bf-8993-0324d0ae1fe2 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:05.540360 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44407386-7a6d-45bf-8993-0324d0ae1fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:05.540630 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44407386-7a6d-45bf-8993-0324d0ae1fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:05.582181 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-44407386-7a6d-45bf-8993-0324d0ae1fe2 None None] MFA Rules not processed for user `5816a1004aa44dea955938edab17ab99`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:05.583398 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-44407386-7a6d-45bf-8993-0324d0ae1fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:05.750038 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1108/2227] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:05.757947 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7be25-b3f8-48aa-91a6-dbd12ab8d7e7 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:05.759938 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-75c7be25-b3f8-48aa-91a6-dbd12ab8d7e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:05.884888 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7be25-b3f8-48aa-91a6-dbd12ab8d7e7 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:05.899756 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456a237-ff67-44c6-9c0a-29b83cc8ab44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:05.904287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456a237-ff67-44c6-9c0a-29b83cc8ab44 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:05.904896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456a237-ff67-44c6-9c0a-29b83cc8ab44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:05.905227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456a237-ff67-44c6-9c0a-29b83cc8ab44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:05.907398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d456a237-ff67-44c6-9c0a-29b83cc8ab44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:06.034360 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d456a237-ff67-44c6-9c0a-29b83cc8ab44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:06.037268 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d456a237-ff67-44c6-9c0a-29b83cc8ab44 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:06.065470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44407386-7a6d-45bf-8993-0324d0ae1fe2 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:06.067205 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-44407386-7a6d-45bf-8993-0324d0ae1fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:06.081108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1120/2228] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:52:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:52:06.085470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:06.243532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44407386-7a6d-45bf-8993-0324d0ae1fe2 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:06.255539 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16266e4f-1a68-423b-affb-b4ca9b22613a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:06.258539 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16266e4f-1a68-423b-affb-b4ca9b22613a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:06.260234 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16266e4f-1a68-423b-affb-b4ca9b22613a admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:06.260755 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16266e4f-1a68-423b-affb-b4ca9b22613a admin admin] PATH_INFO: `/v3/projects/7423a054a3af4b7d8e16f5050a5ea052` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:06.264434 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16266e4f-1a68-423b-affb-b4ca9b22613a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7423a054a3af4b7d8e16f5050a5ea052)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:06.269060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16266e4f-1a68-423b-affb-b4ca9b22613a admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:06.363121 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:06.450079 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-16266e4f-1a68-423b-affb-b4ca9b22613a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7423a054a3af4b7d8e16f5050a5ea052', 'request_id': 'req-16266e4f-1a68-423b-affb-b4ca9b22613a'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:06.459173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-16266e4f-1a68-423b-affb-b4ca9b22613a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7423a054a3af4b7d8e16f5050a5ea052', 'request_id': 'req-16266e4f-1a68-423b-affb-b4ca9b22613a'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:06.546149 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-16266e4f-1a68-423b-affb-b4ca9b22613a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7423a054a3af4b7d8e16f5050a5ea052', 'request_id': 'req-16266e4f-1a68-423b-affb-b4ca9b22613a'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:06.571341 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-16266e4f-1a68-423b-affb-b4ca9b22613a admin admin] The token cache is being invalidate because project 7423a054a3af4b7d8e16f5050a5ea052 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:52:06.571583 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-16266e4f-1a68-423b-affb-b4ca9b22613a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:06.572146 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-16266e4f-1a68-423b-affb-b4ca9b22613a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:06.573951 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1109/2229] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:52:06 2022] DELETE /identity/v3/projects/7423a054a3af4b7d8e16f5050a5ea052 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:52:06.577808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456a237-ff67-44c6-9c0a-29b83cc8ab44 service cinder] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:06.598897 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456a237-ff67-44c6-9c0a-29b83cc8ab44 service cinder] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:06.684720 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83daf5-3db8-416f-b699-6f98ad0e1630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:06.688153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83daf5-3db8-416f-b699-6f98ad0e1630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:06.688153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83daf5-3db8-416f-b699-6f98ad0e1630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:06.688153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83daf5-3db8-416f-b699-6f98ad0e1630 admin admin] PATH_INFO: `/v3/users/fc53af1f5e5c47ff8ca212f5a1d1ef43` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:06.689527 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec83daf5-3db8-416f-b699-6f98ad0e1630 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc53af1f5e5c47ff8ca212f5a1d1ef43)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:06.691807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec83daf5-3db8-416f-b699-6f98ad0e1630 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:07.345319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-ec83daf5-3db8-416f-b699-6f98ad0e1630 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc53af1f5e5c47ff8ca212f5a1d1ef43'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:07.378302 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-ec83daf5-3db8-416f-b699-6f98ad0e1630 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc53af1f5e5c47ff8ca212f5a1d1ef43'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:07.453558 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-ec83daf5-3db8-416f-b699-6f98ad0e1630 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc53af1f5e5c47ff8ca212f5a1d1ef43'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:07.453558 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-ec83daf5-3db8-416f-b699-6f98ad0e1630 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc53af1f5e5c47ff8ca212f5a1d1ef43'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:07.467407 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442367cf-a003-473a-88f4-0729264796a3 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:07.467913 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442367cf-a003-473a-88f4-0729264796a3 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:07.468407 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442367cf-a003-473a-88f4-0729264796a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:07.489579 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-ec83daf5-3db8-416f-b699-6f98ad0e1630 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc53af1f5e5c47ff8ca212f5a1d1ef43'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:07.506072 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1121/2230] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:52:06 2022] DELETE /identity/v3/users/fc53af1f5e5c47ff8ca212f5a1d1ef43 => generated 0 bytes in 928 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:07.537483 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-442367cf-a003-473a-88f4-0729264796a3 None None] MFA Rules not processed for user `83d3494760b84bc1aad951bba11caea3`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:07.548907 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-442367cf-a003-473a-88f4-0729264796a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:07.748437 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1110/2231] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:52:07.760087 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83daf5-3db8-416f-b699-6f98ad0e1630 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:07.763094 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ec83daf5-3db8-416f-b699-6f98ad0e1630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:07.843962 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83daf5-3db8-416f-b699-6f98ad0e1630 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:07.848624 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72cdfc9-bcc0-4898-85b6-685b44382c17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:07.849866 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cdfc9-bcc0-4898-85b6-685b44382c17 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:07.850210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cdfc9-bcc0-4898-85b6-685b44382c17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:07.850800 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72cdfc9-bcc0-4898-85b6-685b44382c17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:07.904870 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72cdfc9-bcc0-4898-85b6-685b44382c17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:07.904870 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72cdfc9-bcc0-4898-85b6-685b44382c17 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:07.912921 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1122/2232] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:52:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:52:08.248906 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442367cf-a003-473a-88f4-0729264796a3 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:08.250510 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-442367cf-a003-473a-88f4-0729264796a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:08.384198 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442367cf-a003-473a-88f4-0729264796a3 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:08.396473 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5256a65-122b-46f0-9f7c-0aeaa9aed47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:08.398109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5256a65-122b-46f0-9f7c-0aeaa9aed47c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:08.398462 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5256a65-122b-46f0-9f7c-0aeaa9aed47c admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:08.398808 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5256a65-122b-46f0-9f7c-0aeaa9aed47c admin admin] PATH_INFO: `/v3/projects/420dccccc18c49cb89708a430be1c82b` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:08.400088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5256a65-122b-46f0-9f7c-0aeaa9aed47c admin admin] RBAC: Authorizing `identity:delete_project(project_id=420dccccc18c49cb89708a430be1c82b)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:08.402196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5256a65-122b-46f0-9f7c-0aeaa9aed47c admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:08.548659 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c5256a65-122b-46f0-9f7c-0aeaa9aed47c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '420dccccc18c49cb89708a430be1c82b', 'request_id': 'req-c5256a65-122b-46f0-9f7c-0aeaa9aed47c'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:08.555813 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c5256a65-122b-46f0-9f7c-0aeaa9aed47c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '420dccccc18c49cb89708a430be1c82b', 'request_id': 'req-c5256a65-122b-46f0-9f7c-0aeaa9aed47c'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:08.620976 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c5256a65-122b-46f0-9f7c-0aeaa9aed47c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '420dccccc18c49cb89708a430be1c82b', 'request_id': 'req-c5256a65-122b-46f0-9f7c-0aeaa9aed47c'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:08.635558 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c5256a65-122b-46f0-9f7c-0aeaa9aed47c admin admin] The token cache is being invalidate because project 420dccccc18c49cb89708a430be1c82b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:52:08.635858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c5256a65-122b-46f0-9f7c-0aeaa9aed47c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:08.636520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c5256a65-122b-46f0-9f7c-0aeaa9aed47c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:08.642823 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1111/2233] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:52:08 2022] DELETE /identity/v3/projects/420dccccc18c49cb89708a430be1c82b => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:08.643207 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc27e43-83bc-47cc-9758-256dd776b7c2 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:08.643506 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc27e43-83bc-47cc-9758-256dd776b7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:08.643792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc27e43-83bc-47cc-9758-256dd776b7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:08.674681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-bbc27e43-83bc-47cc-9758-256dd776b7c2 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:08.684053 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-bbc27e43-83bc-47cc-9758-256dd776b7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:08.764308 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1123/2234] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:08.771575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f765f2-df5a-4a9d-b03f-4f303842a89a None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:08.771575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f765f2-df5a-4a9d-b03f-4f303842a89a None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:08.771575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f765f2-df5a-4a9d-b03f-4f303842a89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:08.802140 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-95f765f2-df5a-4a9d-b03f-4f303842a89a None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:08.804238 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-95f765f2-df5a-4a9d-b03f-4f303842a89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:08.817457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1112/2235] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:08.818501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc27e43-83bc-47cc-9758-256dd776b7c2 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:08.838038 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc27e43-83bc-47cc-9758-256dd776b7c2 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:08.845688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6291d9-95b9-4ec9-8e5b-bf7d2916fbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:08.846505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6291d9-95b9-4ec9-8e5b-bf7d2916fbb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:08.857964 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6291d9-95b9-4ec9-8e5b-bf7d2916fbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:08.857964 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6291d9-95b9-4ec9-8e5b-bf7d2916fbb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:08.857964 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6291d9-95b9-4ec9-8e5b-bf7d2916fbb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:52:08.857964 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6291d9-95b9-4ec9-8e5b-bf7d2916fbb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:08.857964 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6291d9-95b9-4ec9-8e5b-bf7d2916fbb5 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:08.868390 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1124/2236] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:52:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:52:08.869944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f765f2-df5a-4a9d-b03f-4f303842a89a None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:08.884260 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f765f2-df5a-4a9d-b03f-4f303842a89a None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:08.892031 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b4692-d955-4f96-9e48-6d29a3526325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:08.893005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b4692-d955-4f96-9e48-6d29a3526325 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:08.894586 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b4692-d955-4f96-9e48-6d29a3526325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:08.894712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b4692-d955-4f96-9e48-6d29a3526325 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:08.895951 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b4692-d955-4f96-9e48-6d29a3526325 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:08.899199 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b4692-d955-4f96-9e48-6d29a3526325 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:08.935639 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:09.019411 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6291d9-95b9-4ec9-8e5b-bf7d2916fbb5 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:09.032051 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1113/2237] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:52:08 2022] POST /identity/v3/projects => generated 399 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:52:09.032572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6291d9-95b9-4ec9-8e5b-bf7d2916fbb5 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:09.115200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b4692-d955-4f96-9e48-6d29a3526325 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:09.116774 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-3f7b4692-d955-4f96-9e48-6d29a3526325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:09.152857 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf3d7f-a694-4907-a81a-cc29bdad588f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:09.153643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf3d7f-a694-4907-a81a-cc29bdad588f admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:09.154470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf3d7f-a694-4907-a81a-cc29bdad588f admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:09.154470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf3d7f-a694-4907-a81a-cc29bdad588f admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:09.155795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf3d7f-a694-4907-a81a-cc29bdad588f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:09.158387 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf3d7f-a694-4907-a81a-cc29bdad588f admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:09.204553 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b4692-d955-4f96-9e48-6d29a3526325 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:09.210154 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442f04a-65e3-4740-978b-0ff66c87ac8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:09.212851 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442f04a-65e3-4740-978b-0ff66c87ac8b service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:09.212851 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442f04a-65e3-4740-978b-0ff66c87ac8b service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:09.212851 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4442f04a-65e3-4740-978b-0ff66c87ac8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:09.214157 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-4442f04a-65e3-4740-978b-0ff66c87ac8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:09.264701 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1125/2238] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:52:09 2022] POST /identity/v3/users => generated 300 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:52:09.278459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf3d7f-a694-4907-a81a-cc29bdad588f admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:09.293639 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4442f04a-65e3-4740-978b-0ff66c87ac8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:09.295995 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4442f04a-65e3-4740-978b-0ff66c87ac8b service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:09.299944 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf3d7f-a694-4907-a81a-cc29bdad588f admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:09.308938 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087da76-4cdc-412a-b440-20ae59ac8e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:09.309604 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1114/2239] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:52:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:52:09.310325 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087da76-4cdc-412a-b440-20ae59ac8e18 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:09.310619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087da76-4cdc-412a-b440-20ae59ac8e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:09.311789 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087da76-4cdc-412a-b440-20ae59ac8e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:09.312550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d087da76-4cdc-412a-b440-20ae59ac8e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:09.314722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d087da76-4cdc-412a-b440-20ae59ac8e18 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:09.337772 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1126/2240] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:09.340864 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442f04a-65e3-4740-978b-0ff66c87ac8b service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:09.364795 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4442f04a-65e3-4740-978b-0ff66c87ac8b service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:09.373161 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acae37a-ed1d-4874-91aa-71fb4b761156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:09.373161 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acae37a-ed1d-4874-91aa-71fb4b761156 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:09.373161 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acae37a-ed1d-4874-91aa-71fb4b761156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:09.373161 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acae37a-ed1d-4874-91aa-71fb4b761156 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:09.373772 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acae37a-ed1d-4874-91aa-71fb4b761156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:09.376424 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acae37a-ed1d-4874-91aa-71fb4b761156 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:09.390250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1115/2241] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:09 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:09.394915 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087da76-4cdc-412a-b440-20ae59ac8e18 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:09.409248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087da76-4cdc-412a-b440-20ae59ac8e18 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:09.420441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a6e9ea-c8c2-42ba-b483-443c04c5463d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:09.421411 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6e9ea-c8c2-42ba-b483-443c04c5463d admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:09.421700 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6e9ea-c8c2-42ba-b483-443c04c5463d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:09.421983 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6e9ea-c8c2-42ba-b483-443c04c5463d admin admin] PATH_INFO: `/v3/projects/309b1ae86f6b4f73ad15fff37b86abce/users/874794890bb9483e9b66cd8ebe17d412/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:09.446012 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a6e9ea-c8c2-42ba-b483-443c04c5463d admin admin] RBAC: Authorizing `identity:create_grant(project_id=309b1ae86f6b4f73ad15fff37b86abce, user_id=874794890bb9483e9b66cd8ebe17d412, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:09.450465 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a6e9ea-c8c2-42ba-b483-443c04c5463d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:09.516334 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1127/2242] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:52:09 2022] PUT /identity/v3/projects/309b1ae86f6b4f73ad15fff37b86abce/users/874794890bb9483e9b66cd8ebe17d412/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:09.527516 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbb594-04e4-4bdb-9fdc-315f44fa2be9 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:09.527873 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbb594-04e4-4bdb-9fdc-315f44fa2be9 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:09.528085 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adbb594-04e4-4bdb-9fdc-315f44fa2be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:09.569458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-7adbb594-04e4-4bdb-9fdc-315f44fa2be9 None None] MFA Rules not processed for user `874794890bb9483e9b66cd8ebe17d412`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:09.570719 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-7adbb594-04e4-4bdb-9fdc-315f44fa2be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:09.633805 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:09.696604 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1116/2243] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:09 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:09.701241 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a6e9ea-c8c2-42ba-b483-443c04c5463d admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:09.774100 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a6e9ea-c8c2-42ba-b483-443c04c5463d admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:09.784331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39882395-7107-49e3-8edf-cd4a4deed9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:09.785144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39882395-7107-49e3-8edf-cd4a4deed9d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:09.785490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39882395-7107-49e3-8edf-cd4a4deed9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:09.785985 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39882395-7107-49e3-8edf-cd4a4deed9d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:09.786864 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39882395-7107-49e3-8edf-cd4a4deed9d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:09.795216 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39882395-7107-49e3-8edf-cd4a4deed9d1 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:09.884653 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa4a51-4f4a-48f2-a86e-272f0205c54b None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:09.884943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa4a51-4f4a-48f2-a86e-272f0205c54b None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:09.885276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa4a51-4f4a-48f2-a86e-272f0205c54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:09.903338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-f3fa4a51-4f4a-48f2-a86e-272f0205c54b None None] MFA Rules not processed for user `83d3494760b84bc1aad951bba11caea3`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:09.904770 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-f3fa4a51-4f4a-48f2-a86e-272f0205c54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:09.937487 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1128/2244] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:52:09 2022] POST /identity/v3/projects => generated 401 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:52:09.941193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39882395-7107-49e3-8edf-cd4a4deed9d1 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:10.000063 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1117/2245] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:52:10.066574 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39882395-7107-49e3-8edf-cd4a4deed9d1 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:10.075059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8d5ed-3499-4c9a-96d7-084cd76b8f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:10.076696 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8d5ed-3499-4c9a-96d7-084cd76b8f8d admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:10.077177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8d5ed-3499-4c9a-96d7-084cd76b8f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:10.077618 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f8d5ed-3499-4c9a-96d7-084cd76b8f8d admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:10.078572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8d5ed-3499-4c9a-96d7-084cd76b8f8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:10.082708 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f8d5ed-3499-4c9a-96d7-084cd76b8f8d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:10.217252 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1129/2246] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:52:09 2022] POST /identity/v3/users => generated 307 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:52:10.218382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa4a51-4f4a-48f2-a86e-272f0205c54b None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:10.232011 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa4a51-4f4a-48f2-a86e-272f0205c54b None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:10.243123 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47856d-edb4-44f7-8dc8-8c03e58fd2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:10.250418 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47856d-edb4-44f7-8dc8-8c03e58fd2bd admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:10.250418 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47856d-edb4-44f7-8dc8-8c03e58fd2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:10.250418 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e47856d-edb4-44f7-8dc8-8c03e58fd2bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:10.250418 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47856d-edb4-44f7-8dc8-8c03e58fd2bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:10.250418 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e47856d-edb4-44f7-8dc8-8c03e58fd2bd admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:10.264818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1118/2247] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:52:10.268565 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8d5ed-3499-4c9a-96d7-084cd76b8f8d admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:10.282635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f8d5ed-3499-4c9a-96d7-084cd76b8f8d admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:10.301041 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de41b299-6532-407f-9447-215fcb9f36ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:10.301994 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41b299-6532-407f-9447-215fcb9f36ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:10.302287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41b299-6532-407f-9447-215fcb9f36ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:10.302570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41b299-6532-407f-9447-215fcb9f36ae admin admin] PATH_INFO: `/v3/projects/e1acb3dd5c364f53a2c81f81e8c2a9ef/users/2b858b5796ce4df5b2cad41584d0d28e/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:10.345812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de41b299-6532-407f-9447-215fcb9f36ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1acb3dd5c364f53a2c81f81e8c2a9ef, user_id=2b858b5796ce4df5b2cad41584d0d28e, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:10.348622 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de41b299-6532-407f-9447-215fcb9f36ae admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:10.409757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:10.421477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1130/2248] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:52:10 2022] PUT /identity/v3/projects/e1acb3dd5c364f53a2c81f81e8c2a9ef/users/2b858b5796ce4df5b2cad41584d0d28e/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:10.422645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47856d-edb4-44f7-8dc8-8c03e58fd2bd admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:10.435691 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e47856d-edb4-44f7-8dc8-8c03e58fd2bd admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:10.544079 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c417e2-8e72-4ff5-a436-6f386187d8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:10.545800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c417e2-8e72-4ff5-a436-6f386187d8da admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:10.548999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c417e2-8e72-4ff5-a436-6f386187d8da admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:10.548999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c417e2-8e72-4ff5-a436-6f386187d8da admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:10.549986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c417e2-8e72-4ff5-a436-6f386187d8da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:10.551772 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c417e2-8e72-4ff5-a436-6f386187d8da admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:10.570453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1119/2249] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:10.572883 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de41b299-6532-407f-9447-215fcb9f36ae admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:10.590685 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de41b299-6532-407f-9447-215fcb9f36ae admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:10.601887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436baa35-fbc2-4e08-9bd7-036061f0cf69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:10.603312 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436baa35-fbc2-4e08-9bd7-036061f0cf69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:10.603312 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436baa35-fbc2-4e08-9bd7-036061f0cf69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:10.603514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436baa35-fbc2-4e08-9bd7-036061f0cf69 admin admin] PATH_INFO: `/v3/projects/e1acb3dd5c364f53a2c81f81e8c2a9ef/users/2b858b5796ce4df5b2cad41584d0d28e/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:10.605532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436baa35-fbc2-4e08-9bd7-036061f0cf69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1acb3dd5c364f53a2c81f81e8c2a9ef, user_id=2b858b5796ce4df5b2cad41584d0d28e, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:10.608535 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436baa35-fbc2-4e08-9bd7-036061f0cf69 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:10.629480 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1131/2250] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:52:10 2022] PUT /identity/v3/projects/e1acb3dd5c364f53a2c81f81e8c2a9ef/users/2b858b5796ce4df5b2cad41584d0d28e/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:10.649343 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7801eaa-e25a-442f-b6a1-4cdf96b05ecd None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:10.649722 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7801eaa-e25a-442f-b6a1-4cdf96b05ecd None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:10.650026 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7801eaa-e25a-442f-b6a1-4cdf96b05ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:10.705688 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-c7801eaa-e25a-442f-b6a1-4cdf96b05ecd None None] MFA Rules not processed for user `2b858b5796ce4df5b2cad41584d0d28e`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:10.707099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-c7801eaa-e25a-442f-b6a1-4cdf96b05ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:10.852547 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1120/2251] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:10 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:52:10.863849 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436baa35-fbc2-4e08-9bd7-036061f0cf69 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:10.868181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-436baa35-fbc2-4e08-9bd7-036061f0cf69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:10.944653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436baa35-fbc2-4e08-9bd7-036061f0cf69 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:10.947033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc978b-8c09-4f9c-bb85-21cef220e2cb None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:10.947449 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc978b-8c09-4f9c-bb85-21cef220e2cb None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:10.947726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcc978b-8c09-4f9c-bb85-21cef220e2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:10.949610 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86abd1-d488-4969-8db3-004aa93bcc89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:10.953634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86abd1-d488-4969-8db3-004aa93bcc89 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:10.954250 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86abd1-d488-4969-8db3-004aa93bcc89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:10.957669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86abd1-d488-4969-8db3-004aa93bcc89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:10.972298 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e86abd1-d488-4969-8db3-004aa93bcc89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:10.974550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e86abd1-d488-4969-8db3-004aa93bcc89 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:10.994196 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-bfcc978b-8c09-4f9c-bb85-21cef220e2cb None None] MFA Rules not processed for user `3c9aa6527c13466bbe5dce68a3b81ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:10.994680 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1132/2252] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:52:10 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:52:10.994813 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-bfcc978b-8c09-4f9c-bb85-21cef220e2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:11.073767 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1121/2253] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:52:11.133975 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:11.250617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86abd1-d488-4969-8db3-004aa93bcc89 service cinder] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:11.262709 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86abd1-d488-4969-8db3-004aa93bcc89 service cinder] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:11.268653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d419932-4e70-43dd-835c-7f8388924385 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:11.269469 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d419932-4e70-43dd-835c-7f8388924385 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:11.269941 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d419932-4e70-43dd-835c-7f8388924385 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:11.270305 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d419932-4e70-43dd-835c-7f8388924385 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:11.364882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcc978b-8c09-4f9c-bb85-21cef220e2cb None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:11.379264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcc978b-8c09-4f9c-bb85-21cef220e2cb None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:11.383615 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97b19c-5fa9-41f9-96af-9ed7e2b3d038 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:11.384268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97b19c-5fa9-41f9-96af-9ed7e2b3d038 service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:11.384558 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97b19c-5fa9-41f9-96af-9ed7e2b3d038 service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:11.384825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de97b19c-5fa9-41f9-96af-9ed7e2b3d038 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:11.386628 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-de97b19c-5fa9-41f9-96af-9ed7e2b3d038 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:11.387275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d419932-4e70-43dd-835c-7f8388924385 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:11.389196 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d419932-4e70-43dd-835c-7f8388924385 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:11.403774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1133/2254] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:52:11 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:52:11.465500 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97b19c-5fa9-41f9-96af-9ed7e2b3d038 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:11.466957 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de97b19c-5fa9-41f9-96af-9ed7e2b3d038 service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:11.478822 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1122/2255] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:52:11 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:52:12.292912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:12.997310 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:13.832466 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:14.047668 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435849e5-652c-45bf-99a2-0999df0793a2 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:14.047960 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435849e5-652c-45bf-99a2-0999df0793a2 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:14.048412 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435849e5-652c-45bf-99a2-0999df0793a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:14.080833 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-435849e5-652c-45bf-99a2-0999df0793a2 None None] MFA Rules not processed for user `c0f12003099f4f80bb89c2a335d231ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:14.082314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-435849e5-652c-45bf-99a2-0999df0793a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:14.142058 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1134/2256] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:14.745351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:15.411615 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:16.047799 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:16.204024 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:16.706015 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:16.762625 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:17.142555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:17.215302 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:17.216940 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97b19c-5fa9-41f9-96af-9ed7e2b3d038 service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:17.220090 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-de97b19c-5fa9-41f9-96af-9ed7e2b3d038 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:17.247265 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435849e5-652c-45bf-99a2-0999df0793a2 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:17.333479 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de97b19c-5fa9-41f9-96af-9ed7e2b3d038 service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:17.347778 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34d931-6e91-4df1-81a8-9623e59d8117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:17.348894 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34d931-6e91-4df1-81a8-9623e59d8117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:17.349302 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34d931-6e91-4df1-81a8-9623e59d8117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:17.350456 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34d931-6e91-4df1-81a8-9623e59d8117 admin admin] PATH_INFO: `/v3/users/e279760852fd4370b372a09c8148bb8b` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:17.354594 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34d931-6e91-4df1-81a8-9623e59d8117 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e279760852fd4370b372a09c8148bb8b)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:17.359662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed34d931-6e91-4df1-81a8-9623e59d8117 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:17.403793 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435849e5-652c-45bf-99a2-0999df0793a2 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:17.421765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:17.439595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555f552-f876-4a0a-b795-47c5b2665c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:17.442467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555f552-f876-4a0a-b795-47c5b2665c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:17.442957 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555f552-f876-4a0a-b795-47c5b2665c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:17.443457 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555f552-f876-4a0a-b795-47c5b2665c8f admin admin] PATH_INFO: `/v3/users/874794890bb9483e9b66cd8ebe17d412` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:17.445817 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5555f552-f876-4a0a-b795-47c5b2665c8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=874794890bb9483e9b66cd8ebe17d412)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:17.448578 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5555f552-f876-4a0a-b795-47c5b2665c8f admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:17.624603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-ed34d931-6e91-4df1-81a8-9623e59d8117 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e279760852fd4370b372a09c8148bb8b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:17.673776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-ed34d931-6e91-4df1-81a8-9623e59d8117 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e279760852fd4370b372a09c8148bb8b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:17.688498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-5555f552-f876-4a0a-b795-47c5b2665c8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '874794890bb9483e9b66cd8ebe17d412'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:17.705205 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-5555f552-f876-4a0a-b795-47c5b2665c8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '874794890bb9483e9b66cd8ebe17d412'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:17.723189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:17.762209 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-ed34d931-6e91-4df1-81a8-9623e59d8117 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e279760852fd4370b372a09c8148bb8b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:17.763799 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-ed34d931-6e91-4df1-81a8-9623e59d8117 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e279760852fd4370b372a09c8148bb8b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:17.764428 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-ed34d931-6e91-4df1-81a8-9623e59d8117 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e279760852fd4370b372a09c8148bb8b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:17.778663 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1123/2257] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:52:17 2022] DELETE /identity/v3/users/e279760852fd4370b372a09c8148bb8b => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:17.782817 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a05a7-3c8e-4c14-b094-232c38e53892 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:17.783111 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a05a7-3c8e-4c14-b094-232c38e53892 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:17.783394 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a05a7-3c8e-4c14-b094-232c38e53892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:17.801068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-0d2a05a7-3c8e-4c14-b094-232c38e53892 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:17.802386 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-0d2a05a7-3c8e-4c14-b094-232c38e53892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:17.869042 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1124/2258] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:17.878603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a05a7-3c8e-4c14-b094-232c38e53892 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:17.879998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-0d2a05a7-3c8e-4c14-b094-232c38e53892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:17.888723 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-5555f552-f876-4a0a-b795-47c5b2665c8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '874794890bb9483e9b66cd8ebe17d412'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:17.891410 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-5555f552-f876-4a0a-b795-47c5b2665c8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '874794890bb9483e9b66cd8ebe17d412'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:17.900763 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-5555f552-f876-4a0a-b795-47c5b2665c8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '874794890bb9483e9b66cd8ebe17d412'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:17.908659 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1135/2259] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:52:17 2022] DELETE /identity/v3/users/874794890bb9483e9b66cd8ebe17d412 => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:52:17.913293 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decf933-c45b-4e98-9b72-480e53a82803 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:17.913536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decf933-c45b-4e98-9b72-480e53a82803 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:17.913917 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decf933-c45b-4e98-9b72-480e53a82803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:17.933605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-2decf933-c45b-4e98-9b72-480e53a82803 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:17.934805 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-2decf933-c45b-4e98-9b72-480e53a82803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:17.940909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a05a7-3c8e-4c14-b094-232c38e53892 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:18.009905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db1e480-85a3-46b9-9bc8-06771b5fcd36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:18.010443 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1e480-85a3-46b9-9bc8-06771b5fcd36 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:18.011004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1e480-85a3-46b9-9bc8-06771b5fcd36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:18.011175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db1e480-85a3-46b9-9bc8-06771b5fcd36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:18.013378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-3db1e480-85a3-46b9-9bc8-06771b5fcd36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:18.022779 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1136/2260] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:52:18.028323 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db1e480-85a3-46b9-9bc8-06771b5fcd36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:18.029722 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db1e480-85a3-46b9-9bc8-06771b5fcd36 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:18.031143 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2decf933-c45b-4e98-9b72-480e53a82803 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:18.032591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-2decf933-c45b-4e98-9b72-480e53a82803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:18.044189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1125/2261] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:52:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:52:18.057672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2decf933-c45b-4e98-9b72-480e53a82803 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:18.060749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f950c2-c43c-4d3d-8634-17086c2aa4c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:18.061482 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f950c2-c43c-4d3d-8634-17086c2aa4c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:18.061764 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f950c2-c43c-4d3d-8634-17086c2aa4c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:18.062032 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f950c2-c43c-4d3d-8634-17086c2aa4c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:18.086191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f950c2-c43c-4d3d-8634-17086c2aa4c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:18.087922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f950c2-c43c-4d3d-8634-17086c2aa4c8 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:18.099780 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1137/2262] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:52:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:52:18.772012 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db1e480-85a3-46b9-9bc8-06771b5fcd36 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:18.773799 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-3db1e480-85a3-46b9-9bc8-06771b5fcd36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:18.777283 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f950c2-c43c-4d3d-8634-17086c2aa4c8 service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:18.779491 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-90f950c2-c43c-4d3d-8634-17086c2aa4c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:18.795698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db1e480-85a3-46b9-9bc8-06771b5fcd36 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:18.806843 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798571cc-1777-44a2-b47d-70bc7c69c38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:18.808930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798571cc-1777-44a2-b47d-70bc7c69c38b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:18.809472 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798571cc-1777-44a2-b47d-70bc7c69c38b admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:18.810215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798571cc-1777-44a2-b47d-70bc7c69c38b admin admin] PATH_INFO: `/v3/projects/309b1ae86f6b4f73ad15fff37b86abce` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:18.812649 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798571cc-1777-44a2-b47d-70bc7c69c38b admin admin] RBAC: Authorizing `identity:delete_project(project_id=309b1ae86f6b4f73ad15fff37b86abce)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:18.814976 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798571cc-1777-44a2-b47d-70bc7c69c38b admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:18.829606 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f950c2-c43c-4d3d-8634-17086c2aa4c8 service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:18.847271 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ca6e6-7719-4596-84f0-dc2c0ccdd6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:18.849384 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ca6e6-7719-4596-84f0-dc2c0ccdd6f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:18.850774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ca6e6-7719-4596-84f0-dc2c0ccdd6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:18.851109 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ca6e6-7719-4596-84f0-dc2c0ccdd6f2 admin admin] PATH_INFO: `/v3/projects/ad4a9053ff334183a580cd1dbf8eea18` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:18.855792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ca6e6-7719-4596-84f0-dc2c0ccdd6f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad4a9053ff334183a580cd1dbf8eea18)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:18.858658 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ca6e6-7719-4596-84f0-dc2c0ccdd6f2 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:18.905782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:18.958927 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-798571cc-1777-44a2-b47d-70bc7c69c38b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '309b1ae86f6b4f73ad15fff37b86abce', 'request_id': 'req-798571cc-1777-44a2-b47d-70bc7c69c38b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:18.965782 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-798571cc-1777-44a2-b47d-70bc7c69c38b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '309b1ae86f6b4f73ad15fff37b86abce', 'request_id': 'req-798571cc-1777-44a2-b47d-70bc7c69c38b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:18.993362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c73ca6e6-7719-4596-84f0-dc2c0ccdd6f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad4a9053ff334183a580cd1dbf8eea18', 'request_id': 'req-c73ca6e6-7719-4596-84f0-dc2c0ccdd6f2'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:18.999755 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c73ca6e6-7719-4596-84f0-dc2c0ccdd6f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad4a9053ff334183a580cd1dbf8eea18', 'request_id': 'req-c73ca6e6-7719-4596-84f0-dc2c0ccdd6f2'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:19.007468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c73ca6e6-7719-4596-84f0-dc2c0ccdd6f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad4a9053ff334183a580cd1dbf8eea18', 'request_id': 'req-c73ca6e6-7719-4596-84f0-dc2c0ccdd6f2'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:19.059629 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-798571cc-1777-44a2-b47d-70bc7c69c38b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '309b1ae86f6b4f73ad15fff37b86abce', 'request_id': 'req-798571cc-1777-44a2-b47d-70bc7c69c38b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:19.076681 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-798571cc-1777-44a2-b47d-70bc7c69c38b admin admin] The token cache is being invalidate because project 309b1ae86f6b4f73ad15fff37b86abce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:52:19.076926 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-798571cc-1777-44a2-b47d-70bc7c69c38b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:19.077692 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-798571cc-1777-44a2-b47d-70bc7c69c38b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:19.082326 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1126/2263] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:52:18 2022] DELETE /identity/v3/projects/309b1ae86f6b4f73ad15fff37b86abce => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:19.083928 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798571cc-1777-44a2-b47d-70bc7c69c38b admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:19.086549 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-798571cc-1777-44a2-b47d-70bc7c69c38b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:19.165951 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c73ca6e6-7719-4596-84f0-dc2c0ccdd6f2 admin admin] The token cache is being invalidate because project ad4a9053ff334183a580cd1dbf8eea18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:52:19.165951 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c73ca6e6-7719-4596-84f0-dc2c0ccdd6f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:19.166920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c73ca6e6-7719-4596-84f0-dc2c0ccdd6f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:19.168588 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1138/2264] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:52:18 2022] DELETE /identity/v3/projects/ad4a9053ff334183a580cd1dbf8eea18 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:52:19.172099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:19.174974 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ca6e6-7719-4596-84f0-dc2c0ccdd6f2 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:19.179099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-c73ca6e6-7719-4596-84f0-dc2c0ccdd6f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:19.262986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798571cc-1777-44a2-b47d-70bc7c69c38b admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:19.368471 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf460c3-dd11-4fcf-a850-d52a7c91c9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:19.370090 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf460c3-dd11-4fcf-a850-d52a7c91c9c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:19.370090 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf460c3-dd11-4fcf-a850-d52a7c91c9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:19.370090 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf460c3-dd11-4fcf-a850-d52a7c91c9c8 admin admin] PATH_INFO: `/v3/users/2b858b5796ce4df5b2cad41584d0d28e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:19.371938 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf460c3-dd11-4fcf-a850-d52a7c91c9c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b858b5796ce4df5b2cad41584d0d28e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:19.374166 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf460c3-dd11-4fcf-a850-d52a7c91c9c8 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:19.380485 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ca6e6-7719-4596-84f0-dc2c0ccdd6f2 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:19.387019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9860c3f1-5875-4a28-b0a5-8368bdfb43ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:19.389624 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860c3f1-5875-4a28-b0a5-8368bdfb43ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:19.397485 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860c3f1-5875-4a28-b0a5-8368bdfb43ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:19.398074 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9860c3f1-5875-4a28-b0a5-8368bdfb43ab admin admin] PATH_INFO: `/v3/users/9c31944aa13e4ee186dc6594ed198ad6` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:19.400577 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9860c3f1-5875-4a28-b0a5-8368bdfb43ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c31944aa13e4ee186dc6594ed198ad6)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:19.403142 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9860c3f1-5875-4a28-b0a5-8368bdfb43ab admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:19.661951 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-9860c3f1-5875-4a28-b0a5-8368bdfb43ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c31944aa13e4ee186dc6594ed198ad6'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:19.670750 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1bf460c3-dd11-4fcf-a850-d52a7c91c9c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b858b5796ce4df5b2cad41584d0d28e'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:19.684664 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-9860c3f1-5875-4a28-b0a5-8368bdfb43ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c31944aa13e4ee186dc6594ed198ad6'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:19.692101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1bf460c3-dd11-4fcf-a850-d52a7c91c9c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b858b5796ce4df5b2cad41584d0d28e'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:19.728221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:19.785823 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-9860c3f1-5875-4a28-b0a5-8368bdfb43ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c31944aa13e4ee186dc6594ed198ad6'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:19.787850 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-9860c3f1-5875-4a28-b0a5-8368bdfb43ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c31944aa13e4ee186dc6594ed198ad6'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:19.797259 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-9860c3f1-5875-4a28-b0a5-8368bdfb43ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c31944aa13e4ee186dc6594ed198ad6'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:19.806060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1139/2265] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:52:19 2022] DELETE /identity/v3/users/9c31944aa13e4ee186dc6594ed198ad6 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:19.848457 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed52b0-f94f-488f-82a7-e0618ad11d61 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:19.848788 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed52b0-f94f-488f-82a7-e0618ad11d61 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:19.849119 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed52b0-f94f-488f-82a7-e0618ad11d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:19.871055 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1bf460c3-dd11-4fcf-a850-d52a7c91c9c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b858b5796ce4df5b2cad41584d0d28e'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:19.872383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1bf460c3-dd11-4fcf-a850-d52a7c91c9c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b858b5796ce4df5b2cad41584d0d28e'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:19.873340 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1bf460c3-dd11-4fcf-a850-d52a7c91c9c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b858b5796ce4df5b2cad41584d0d28e'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:19.873790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-a3ed52b0-f94f-488f-82a7-e0618ad11d61 None None] MFA Rules not processed for user `c700b0adb9f54947a88c1baa86accb9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:19.876241 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-a3ed52b0-f94f-488f-82a7-e0618ad11d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:19.901648 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1127/2266] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:52:19 2022] DELETE /identity/v3/users/2b858b5796ce4df5b2cad41584d0d28e => generated 0 bytes in 818 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:19.982272 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1140/2267] 10.222.0.13 () {58 vars in 1009 bytes} [Fri May 27 19:52:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:52:19.999201 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf460c3-dd11-4fcf-a850-d52a7c91c9c8 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:20.000821 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-1bf460c3-dd11-4fcf-a850-d52a7c91c9c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:20.128171 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf460c3-dd11-4fcf-a850-d52a7c91c9c8 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:20.137158 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258f14a-af7c-4a43-a2a5-33951532e8a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37bc6d0abb454de19e1ab2f46bf4a593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:20.139991 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258f14a-af7c-4a43-a2a5-33951532e8a2 service swift] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:20.140532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258f14a-af7c-4a43-a2a5-33951532e8a2 service swift] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:20.140973 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e258f14a-af7c-4a43-a2a5-33951532e8a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:20.269945 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e258f14a-af7c-4a43-a2a5-33951532e8a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:20.276290 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e258f14a-af7c-4a43-a2a5-33951532e8a2 service swift] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:20.292290 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1128/2268] 10.222.0.13 () {60 vars in 1384 bytes} [Fri May 27 19:52:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 19:52:20.306590 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:21.073153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d6032-1f8f-4207-a010-9c450032418e None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:21.074323 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d6032-1f8f-4207-a010-9c450032418e None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:21.074617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8d6032-1f8f-4207-a010-9c450032418e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:21.135953 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-df8d6032-1f8f-4207-a010-9c450032418e None None] MFA Rules not processed for user `c700b0adb9f54947a88c1baa86accb9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:21.139744 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-df8d6032-1f8f-4207-a010-9c450032418e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:21.168762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1141/2269] 10.222.0.13 () {58 vars in 1009 bytes} [Fri May 27 19:52:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:21.187018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258f14a-af7c-4a43-a2a5-33951532e8a2 service swift] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:21.207464 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e258f14a-af7c-4a43-a2a5-33951532e8a2 service swift] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:21.229021 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd3049-e4a1-4df0-9c08-a16182e1d29c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37bc6d0abb454de19e1ab2f46bf4a593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:21.229782 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd3049-e4a1-4df0-9c08-a16182e1d29c service swift] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:21.230694 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd3049-e4a1-4df0-9c08-a16182e1d29c service swift] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:21.230694 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd3049-e4a1-4df0-9c08-a16182e1d29c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:21.251270 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd3049-e4a1-4df0-9c08-a16182e1d29c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:21.253781 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd3049-e4a1-4df0-9c08-a16182e1d29c service swift] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:21.282052 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1129/2270] 10.222.0.13 () {60 vars in 1384 bytes} [Fri May 27 19:52:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 19:52:21.420358 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d6032-1f8f-4207-a010-9c450032418e None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:21.422464 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-df8d6032-1f8f-4207-a010-9c450032418e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:21.609404 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8d6032-1f8f-4207-a010-9c450032418e None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:21.625741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975fa20-c2d3-40ae-afa3-5aab14c909dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:21.626742 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975fa20-c2d3-40ae-afa3-5aab14c909dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:21.627131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975fa20-c2d3-40ae-afa3-5aab14c909dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:21.628604 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975fa20-c2d3-40ae-afa3-5aab14c909dd admin admin] PATH_INFO: `/v3/projects/e1acb3dd5c364f53a2c81f81e8c2a9ef` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:21.633158 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975fa20-c2d3-40ae-afa3-5aab14c909dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1acb3dd5c364f53a2c81f81e8c2a9ef)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:21.637644 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975fa20-c2d3-40ae-afa3-5aab14c909dd admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:21.717059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:21.795555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-5975fa20-c2d3-40ae-afa3-5aab14c909dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1acb3dd5c364f53a2c81f81e8c2a9ef', 'request_id': 'req-5975fa20-c2d3-40ae-afa3-5aab14c909dd'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:21.829106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-5975fa20-c2d3-40ae-afa3-5aab14c909dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1acb3dd5c364f53a2c81f81e8c2a9ef', 'request_id': 'req-5975fa20-c2d3-40ae-afa3-5aab14c909dd'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:21.845251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-5975fa20-c2d3-40ae-afa3-5aab14c909dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1acb3dd5c364f53a2c81f81e8c2a9ef', 'request_id': 'req-5975fa20-c2d3-40ae-afa3-5aab14c909dd'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:21.865182 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:21.896243 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd3049-e4a1-4df0-9c08-a16182e1d29c service swift] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:21.898505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-87fd3049-e4a1-4df0-9c08-a16182e1d29c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:21.932779 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd3049-e4a1-4df0-9c08-a16182e1d29c service swift] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:21.984572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-5975fa20-c2d3-40ae-afa3-5aab14c909dd admin admin] The token cache is being invalidate because project e1acb3dd5c364f53a2c81f81e8c2a9ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:52:21.985102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-5975fa20-c2d3-40ae-afa3-5aab14c909dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:21.987097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-5975fa20-c2d3-40ae-afa3-5aab14c909dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:21.995431 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1142/2271] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:52:21 2022] DELETE /identity/v3/projects/e1acb3dd5c364f53a2c81f81e8c2a9ef => generated 0 bytes in 572 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:21.998922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f1026-8d1c-4178-8593-7aac36e725d1 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:22.002322 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f1026-8d1c-4178-8593-7aac36e725d1 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:22.002876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413f1026-8d1c-4178-8593-7aac36e725d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:22.062889 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-413f1026-8d1c-4178-8593-7aac36e725d1 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:22.069282 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-413f1026-8d1c-4178-8593-7aac36e725d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:22.202752 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d0749-cdb0-4a0f-a587-c9b8fe582299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:22.204360 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d0749-cdb0-4a0f-a587-c9b8fe582299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:22.205923 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d0749-cdb0-4a0f-a587-c9b8fe582299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:22.205923 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14d0749-cdb0-4a0f-a587-c9b8fe582299 admin admin] PATH_INFO: `/v3/projects/56ecbb75334f465c97d12918f529283b` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:22.208924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d0749-cdb0-4a0f-a587-c9b8fe582299 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56ecbb75334f465c97d12918f529283b)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:22.214500 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14d0749-cdb0-4a0f-a587-c9b8fe582299 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:22.221909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1143/2272] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:22.223908 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66de99-e03a-4bc8-a2f3-9c5108dba990 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:22.224307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66de99-e03a-4bc8-a2f3-9c5108dba990 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:22.224745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a66de99-e03a-4bc8-a2f3-9c5108dba990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:22.286114 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-8a66de99-e03a-4bc8-a2f3-9c5108dba990 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:22.294290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-8a66de99-e03a-4bc8-a2f3-9c5108dba990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:22.343985 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1144/2273] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:22.344808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66de99-e03a-4bc8-a2f3-9c5108dba990 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:22.402138 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a66de99-e03a-4bc8-a2f3-9c5108dba990 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:22.422533 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b972dde-55c4-496c-80ed-f244dd16b25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:22.423375 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b972dde-55c4-496c-80ed-f244dd16b25f admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:22.423761 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b972dde-55c4-496c-80ed-f244dd16b25f admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:22.424148 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b972dde-55c4-496c-80ed-f244dd16b25f admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:22.425010 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b972dde-55c4-496c-80ed-f244dd16b25f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:52:22.425425 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b972dde-55c4-496c-80ed-f244dd16b25f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:22.427282 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b972dde-55c4-496c-80ed-f244dd16b25f admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:22.462124 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1145/2274] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:52:22.475871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b972dde-55c4-496c-80ed-f244dd16b25f admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:22.490250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d14d0749-cdb0-4a0f-a587-c9b8fe582299 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56ecbb75334f465c97d12918f529283b', 'request_id': 'req-d14d0749-cdb0-4a0f-a587-c9b8fe582299'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:22.503937 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b972dde-55c4-496c-80ed-f244dd16b25f admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:22.547173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d14d0749-cdb0-4a0f-a587-c9b8fe582299 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56ecbb75334f465c97d12918f529283b', 'request_id': 'req-d14d0749-cdb0-4a0f-a587-c9b8fe582299'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:22.620380 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe7875-674d-4ebb-9055-847e3697092a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:22.621520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe7875-674d-4ebb-9055-847e3697092a admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:22.622071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe7875-674d-4ebb-9055-847e3697092a admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:22.622571 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fe7875-674d-4ebb-9055-847e3697092a admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:22.623512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe7875-674d-4ebb-9055-847e3697092a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:22.626159 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fe7875-674d-4ebb-9055-847e3697092a admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:22.664930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d14d0749-cdb0-4a0f-a587-c9b8fe582299 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56ecbb75334f465c97d12918f529283b', 'request_id': 'req-d14d0749-cdb0-4a0f-a587-c9b8fe582299'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:22.706634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d14d0749-cdb0-4a0f-a587-c9b8fe582299 admin admin] The token cache is being invalidate because project 56ecbb75334f465c97d12918f529283b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:52:22.707517 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d14d0749-cdb0-4a0f-a587-c9b8fe582299 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:22.709636 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d14d0749-cdb0-4a0f-a587-c9b8fe582299 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:22.712133 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1130/2275] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:52:21 2022] DELETE /identity/v3/projects/56ecbb75334f465c97d12918f529283b => generated 0 bytes in 816 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:52:22.715980 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d0749-cdb0-4a0f-a587-c9b8fe582299 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:22.718748 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d14d0749-cdb0-4a0f-a587-c9b8fe582299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:22.735355 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14d0749-cdb0-4a0f-a587-c9b8fe582299 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:22.746821 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca168d-c4ee-41de-8510-5eaf61ce04a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:22.752349 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1146/2276] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:52:22 2022] POST /identity/v3/projects => generated 381 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:52:22.752789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca168d-c4ee-41de-8510-5eaf61ce04a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:22.753266 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca168d-c4ee-41de-8510-5eaf61ce04a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:22.753741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca168d-c4ee-41de-8510-5eaf61ce04a7 admin admin] PATH_INFO: `/v3/users/5816a1004aa44dea955938edab17ab99` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:22.756241 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca168d-c4ee-41de-8510-5eaf61ce04a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5816a1004aa44dea955938edab17ab99)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:22.758148 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe7875-674d-4ebb-9055-847e3697092a admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:22.760680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca168d-c4ee-41de-8510-5eaf61ce04a7 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:22.767312 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-02fe7875-674d-4ebb-9055-847e3697092a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:22.908893 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fe7875-674d-4ebb-9055-847e3697092a admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:22.922921 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dffb1f0-af73-475f-a826-add578646135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:22.922921 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffb1f0-af73-475f-a826-add578646135 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:22.923328 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffb1f0-af73-475f-a826-add578646135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:22.923709 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffb1f0-af73-475f-a826-add578646135 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:22.924662 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dffb1f0-af73-475f-a826-add578646135 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:22.927473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dffb1f0-af73-475f-a826-add578646135 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:23.027336 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-03ca168d-c4ee-41de-8510-5eaf61ce04a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5816a1004aa44dea955938edab17ab99'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:23.073998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-03ca168d-c4ee-41de-8510-5eaf61ce04a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5816a1004aa44dea955938edab17ab99'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:23.101553 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1147/2277] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:52:22 2022] POST /identity/v3/users => generated 291 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:52:23.106058 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dffb1f0-af73-475f-a826-add578646135 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:23.143462 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dffb1f0-af73-475f-a826-add578646135 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:23.157392 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0446e28-fbe8-4aef-9161-8a8c6700b9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:23.158513 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0446e28-fbe8-4aef-9161-8a8c6700b9bc admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:23.159029 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0446e28-fbe8-4aef-9161-8a8c6700b9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:23.159513 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0446e28-fbe8-4aef-9161-8a8c6700b9bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:23.160462 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0446e28-fbe8-4aef-9161-8a8c6700b9bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:23.162943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0446e28-fbe8-4aef-9161-8a8c6700b9bc admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:23.177425 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1148/2278] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:23.178781 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0446e28-fbe8-4aef-9161-8a8c6700b9bc admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:23.190027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-03ca168d-c4ee-41de-8510-5eaf61ce04a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5816a1004aa44dea955938edab17ab99'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:23.192755 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-03ca168d-c4ee-41de-8510-5eaf61ce04a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5816a1004aa44dea955938edab17ab99'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:23.193823 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-03ca168d-c4ee-41de-8510-5eaf61ce04a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5816a1004aa44dea955938edab17ab99'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:23.203636 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0446e28-fbe8-4aef-9161-8a8c6700b9bc admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:23.212594 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1131/2279] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:52:22 2022] DELETE /identity/v3/users/5816a1004aa44dea955938edab17ab99 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:23.216580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0fc774-936c-4dca-82fa-22514f628299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:23.218092 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0fc774-936c-4dca-82fa-22514f628299 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:23.218592 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0fc774-936c-4dca-82fa-22514f628299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:23.219096 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0fc774-936c-4dca-82fa-22514f628299 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:23.220041 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0fc774-936c-4dca-82fa-22514f628299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:23.221904 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0fc774-936c-4dca-82fa-22514f628299 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:23.241764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca168d-c4ee-41de-8510-5eaf61ce04a7 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:23.243509 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1149/2280] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:23 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:23.244796 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-03ca168d-c4ee-41de-8510-5eaf61ce04a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:23.514962 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca168d-c4ee-41de-8510-5eaf61ce04a7 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:23.528289 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b3b85-a911-44b7-bbe1-f7310c46a9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:23.529297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b3b85-a911-44b7-bbe1-f7310c46a9f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:23.529830 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b3b85-a911-44b7-bbe1-f7310c46a9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:23.530124 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44b3b85-a911-44b7-bbe1-f7310c46a9f0 admin admin] PATH_INFO: `/v3/projects/765b703e9d41448cad65d1d1e45418ea/users/836484879a494ff98737ae0d3980e886/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:23.591058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b3b85-a911-44b7-bbe1-f7310c46a9f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=765b703e9d41448cad65d1d1e45418ea, user_id=836484879a494ff98737ae0d3980e886, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:23.594086 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44b3b85-a911-44b7-bbe1-f7310c46a9f0 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:24.033734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1132/2281] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:52:23 2022] PUT /identity/v3/projects/765b703e9d41448cad65d1d1e45418ea/users/836484879a494ff98737ae0d3980e886/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 792 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:24.045568 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863137c-dfba-4246-a252-95d8d57a2763 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:24.048841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863137c-dfba-4246-a252-95d8d57a2763 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:24.049313 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a863137c-dfba-4246-a252-95d8d57a2763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:24.058150 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:24.190030 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-a863137c-dfba-4246-a252-95d8d57a2763 None None] MFA Rules not processed for user `836484879a494ff98737ae0d3980e886`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:24.197695 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-a863137c-dfba-4246-a252-95d8d57a2763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:24.364900 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1150/2282] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:24.377752 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b3b85-a911-44b7-bbe1-f7310c46a9f0 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:24.462064 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:24.548064 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44b3b85-a911-44b7-bbe1-f7310c46a9f0 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:24.558464 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ee767-b042-4f64-a850-920771e026ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:24.559691 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ee767-b042-4f64-a850-920771e026ab admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:24.560154 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ee767-b042-4f64-a850-920771e026ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:24.560665 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ee767-b042-4f64-a850-920771e026ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:24.563836 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ee767-b042-4f64-a850-920771e026ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:24.569215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ee767-b042-4f64-a850-920771e026ab admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:24.714904 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1133/2283] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:52:24 2022] POST /identity/v3/projects => generated 381 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:52:24.716700 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863137c-dfba-4246-a252-95d8d57a2763 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:24.738340 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a863137c-dfba-4246-a252-95d8d57a2763 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:24.829307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249749f1-82aa-4021-b1b8-f735bdd3d3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:24.830005 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249749f1-82aa-4021-b1b8-f735bdd3d3d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:24.830330 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249749f1-82aa-4021-b1b8-f735bdd3d3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:24.830650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249749f1-82aa-4021-b1b8-f735bdd3d3d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:24.831401 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249749f1-82aa-4021-b1b8-f735bdd3d3d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:24.841352 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249749f1-82aa-4021-b1b8-f735bdd3d3d0 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:24.983940 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1151/2284] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:52:24 2022] POST /identity/v3/users => generated 297 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:52:24.988479 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ee767-b042-4f64-a850-920771e026ab admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:25.010382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ee767-b042-4f64-a850-920771e026ab admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:25.025665 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4dfc63-d96d-42d4-8253-e325a5567889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:25.026592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4dfc63-d96d-42d4-8253-e325a5567889 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:25.026935 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4dfc63-d96d-42d4-8253-e325a5567889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:25.027218 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4dfc63-d96d-42d4-8253-e325a5567889 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:25.027986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4dfc63-d96d-42d4-8253-e325a5567889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:25.032910 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4dfc63-d96d-42d4-8253-e325a5567889 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:25.065160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:25.074954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249749f1-82aa-4021-b1b8-f735bdd3d3d0 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:25.076159 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1134/2285] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:25.095588 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249749f1-82aa-4021-b1b8-f735bdd3d3d0 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:25.111431 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05452af9-6e86-40bb-afdc-a55b5ab5a51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:25.111431 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05452af9-6e86-40bb-afdc-a55b5ab5a51f admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:25.111431 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05452af9-6e86-40bb-afdc-a55b5ab5a51f admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:25.111431 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05452af9-6e86-40bb-afdc-a55b5ab5a51f admin admin] PATH_INFO: `/v3/projects/174102b431064258965e637360d32c86/users/27e27a4d52494b0cbd30c33c9405b937/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:25.181694 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05452af9-6e86-40bb-afdc-a55b5ab5a51f admin admin] RBAC: Authorizing `identity:create_grant(project_id=174102b431064258965e637360d32c86, user_id=27e27a4d52494b0cbd30c33c9405b937, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:25.184887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05452af9-6e86-40bb-afdc-a55b5ab5a51f admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:25.338291 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1152/2286] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:52:25 2022] PUT /identity/v3/projects/174102b431064258965e637360d32c86/users/27e27a4d52494b0cbd30c33c9405b937/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:52:25.342620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4dfc63-d96d-42d4-8253-e325a5567889 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:25.356202 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4dfc63-d96d-42d4-8253-e325a5567889 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:25.561531 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cda917-efda-4021-96ff-9bd9108f69c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:25.562235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cda917-efda-4021-96ff-9bd9108f69c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:25.567153 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cda917-efda-4021-96ff-9bd9108f69c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:25.567464 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cda917-efda-4021-96ff-9bd9108f69c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:25.569063 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cda917-efda-4021-96ff-9bd9108f69c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:25.573025 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cda917-efda-4021-96ff-9bd9108f69c4 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:25.588407 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1135/2287] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:25.594147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05452af9-6e86-40bb-afdc-a55b5ab5a51f admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:25.618586 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05452af9-6e86-40bb-afdc-a55b5ab5a51f admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:25.640478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cda917-efda-4021-96ff-9bd9108f69c4 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:25.647316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d7cda917-efda-4021-96ff-9bd9108f69c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:25.659109 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dc38f3-0260-49b4-89a3-b53f4ad63cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:25.660040 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc38f3-0260-49b4-89a3-b53f4ad63cff admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:25.660331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc38f3-0260-49b4-89a3-b53f4ad63cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:25.660645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dc38f3-0260-49b4-89a3-b53f4ad63cff admin admin] PATH_INFO: `/v3/projects/174102b431064258965e637360d32c86/users/27e27a4d52494b0cbd30c33c9405b937/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:25.667689 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc38f3-0260-49b4-89a3-b53f4ad63cff admin admin] RBAC: Authorizing `identity:create_grant(project_id=174102b431064258965e637360d32c86, user_id=27e27a4d52494b0cbd30c33c9405b937, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:25.672052 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dc38f3-0260-49b4-89a3-b53f4ad63cff admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:25.690027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cda917-efda-4021-96ff-9bd9108f69c4 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:25.707361 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a229a5-9d70-4072-9017-f8b0c18c917c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:25.708183 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a229a5-9d70-4072-9017-f8b0c18c917c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:25.708475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a229a5-9d70-4072-9017-f8b0c18c917c admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:25.708759 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a229a5-9d70-4072-9017-f8b0c18c917c admin admin] PATH_INFO: `/v3/projects/973f04b8ad984f66b8825e408abc005c` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:25.721288 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a229a5-9d70-4072-9017-f8b0c18c917c admin admin] RBAC: Authorizing `identity:delete_project(project_id=973f04b8ad984f66b8825e408abc005c)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:25.723230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a229a5-9d70-4072-9017-f8b0c18c917c admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:25.735922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1153/2288] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:52:25 2022] PUT /identity/v3/projects/174102b431064258965e637360d32c86/users/27e27a4d52494b0cbd30c33c9405b937/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:25.748304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b26f2a-3e40-4251-affa-9627d5af07eb None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:25.748916 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b26f2a-3e40-4251-affa-9627d5af07eb None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:25.749401 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b26f2a-3e40-4251-affa-9627d5af07eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:25.780729 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-d3b26f2a-3e40-4251-affa-9627d5af07eb None None] MFA Rules not processed for user `27e27a4d52494b0cbd30c33c9405b937`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:25.782587 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d3b26f2a-3e40-4251-affa-9627d5af07eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:25.917851 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1154/2289] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:25.938111 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b26f2a-3e40-4251-affa-9627d5af07eb None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:25.943893 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d3b26f2a-3e40-4251-affa-9627d5af07eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:26.013095 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b1a229a5-9d70-4072-9017-f8b0c18c917c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '973f04b8ad984f66b8825e408abc005c', 'request_id': 'req-b1a229a5-9d70-4072-9017-f8b0c18c917c'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:26.025620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b1a229a5-9d70-4072-9017-f8b0c18c917c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '973f04b8ad984f66b8825e408abc005c', 'request_id': 'req-b1a229a5-9d70-4072-9017-f8b0c18c917c'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:26.037964 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b26f2a-3e40-4251-affa-9627d5af07eb None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:26.060010 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:26.094594 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b1a229a5-9d70-4072-9017-f8b0c18c917c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '973f04b8ad984f66b8825e408abc005c', 'request_id': 'req-b1a229a5-9d70-4072-9017-f8b0c18c917c'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:26.116984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b1a229a5-9d70-4072-9017-f8b0c18c917c admin admin] The token cache is being invalidate because project 973f04b8ad984f66b8825e408abc005c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:52:26.117308 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b1a229a5-9d70-4072-9017-f8b0c18c917c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:26.118034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b1a229a5-9d70-4072-9017-f8b0c18c917c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:26.123174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1136/2290] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:52:25 2022] DELETE /identity/v3/projects/973f04b8ad984f66b8825e408abc005c => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:26.124486 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0bacd-9a16-47ad-8b3a-a75d57e9d0c6 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:26.125478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0bacd-9a16-47ad-8b3a-a75d57e9d0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:26.125740 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0bacd-9a16-47ad-8b3a-a75d57e9d0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:26.135205 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:26.142568 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28cf10-ae53-4962-8f05-beb3eab4faba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:26.143297 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28cf10-ae53-4962-8f05-beb3eab4faba service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:26.143586 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28cf10-ae53-4962-8f05-beb3eab4faba service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:26.143933 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28cf10-ae53-4962-8f05-beb3eab4faba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:26.145896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-6d28cf10-ae53-4962-8f05-beb3eab4faba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:26.148497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-a0a0bacd-9a16-47ad-8b3a-a75d57e9d0c6 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:26.149899 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-a0a0bacd-9a16-47ad-8b3a-a75d57e9d0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:26.242094 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28cf10-ae53-4962-8f05-beb3eab4faba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:26.243888 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28cf10-ae53-4962-8f05-beb3eab4faba service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:26.256797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1155/2291] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:52:25 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:52:26.303566 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1137/2292] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:26.310519 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd9601-47a9-4b19-86d0-2c273caea81e None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:26.313573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd9601-47a9-4b19-86d0-2c273caea81e None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:26.313573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd9601-47a9-4b19-86d0-2c273caea81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:26.345467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-46bd9601-47a9-4b19-86d0-2c273caea81e None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:26.346761 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-46bd9601-47a9-4b19-86d0-2c273caea81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:26.371787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1156/2293] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:26.380437 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0bacd-9a16-47ad-8b3a-a75d57e9d0c6 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:26.397465 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0bacd-9a16-47ad-8b3a-a75d57e9d0c6 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:26.404043 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deafa70-8a9f-4421-a965-69b30a73118b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:26.405084 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deafa70-8a9f-4421-a965-69b30a73118b admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:26.405575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deafa70-8a9f-4421-a965-69b30a73118b admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:26.406007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deafa70-8a9f-4421-a965-69b30a73118b admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:26.407050 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deafa70-8a9f-4421-a965-69b30a73118b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:52:26.407683 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deafa70-8a9f-4421-a965-69b30a73118b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:26.409570 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5deafa70-8a9f-4421-a965-69b30a73118b admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:26.427023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1138/2294] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:52:26.434682 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd9601-47a9-4b19-86d0-2c273caea81e None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:26.446352 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd9601-47a9-4b19-86d0-2c273caea81e None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:26.468528 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f324e0-76fa-4904-a615-550b071d8ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:26.473815 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f324e0-76fa-4904-a615-550b071d8ef6 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:26.474913 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f324e0-76fa-4904-a615-550b071d8ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:26.474913 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f324e0-76fa-4904-a615-550b071d8ef6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:26.475768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f324e0-76fa-4904-a615-550b071d8ef6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:26.484224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f324e0-76fa-4904-a615-550b071d8ef6 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:26.557931 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deafa70-8a9f-4421-a965-69b30a73118b admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:26.562739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-5deafa70-8a9f-4421-a965-69b30a73118b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:26.601895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deafa70-8a9f-4421-a965-69b30a73118b admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:26.616996 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b85cc-7d87-442a-9cc8-fde37ac317b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:26.616996 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b85cc-7d87-442a-9cc8-fde37ac317b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:26.616996 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b85cc-7d87-442a-9cc8-fde37ac317b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:26.616996 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9b85cc-7d87-442a-9cc8-fde37ac317b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:26.617490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-cb9b85cc-7d87-442a-9cc8-fde37ac317b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:26.643629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1157/2295] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:52:26 2022] POST /identity/v3/projects => generated 367 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:52:26.648543 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f324e0-76fa-4904-a615-550b071d8ef6 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:26.680035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b85cc-7d87-442a-9cc8-fde37ac317b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:26.681514 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9b85cc-7d87-442a-9cc8-fde37ac317b3 service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:26.690435 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1139/2296] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:52:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:52:26.715292 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f324e0-76fa-4904-a615-550b071d8ef6 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:26.721845 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ffb7d-bb43-4456-b702-8f56d3e10b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:26.721845 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ffb7d-bb43-4456-b702-8f56d3e10b7d admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:26.721845 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ffb7d-bb43-4456-b702-8f56d3e10b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:26.721845 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ffb7d-bb43-4456-b702-8f56d3e10b7d admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:26.722539 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3ffb7d-bb43-4456-b702-8f56d3e10b7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:26.724531 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3ffb7d-bb43-4456-b702-8f56d3e10b7d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:26.961811 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b85cc-7d87-442a-9cc8-fde37ac317b3 service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:26.963321 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-cb9b85cc-7d87-442a-9cc8-fde37ac317b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:26.978200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9b85cc-7d87-442a-9cc8-fde37ac317b3 service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:26.982843 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553f6e1-21b8-44d8-b477-88f6c363f7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:26.983892 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553f6e1-21b8-44d8-b477-88f6c363f7a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:26.989855 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553f6e1-21b8-44d8-b477-88f6c363f7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:26.989855 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553f6e1-21b8-44d8-b477-88f6c363f7a7 admin admin] PATH_INFO: `/v3/users/ba002a25cfaa47cbb738dc1470106c7d` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:26.989855 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4553f6e1-21b8-44d8-b477-88f6c363f7a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba002a25cfaa47cbb738dc1470106c7d)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:26.989855 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4553f6e1-21b8-44d8-b477-88f6c363f7a7 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:27.087146 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1158/2297] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:52:26 2022] POST /identity/v3/users => generated 284 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:52:27.090116 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ffb7d-bb43-4456-b702-8f56d3e10b7d admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:27.107656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ffb7d-bb43-4456-b702-8f56d3e10b7d admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:27.126496 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab2e87-7446-4fa2-8b2b-ca407ec2dfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:27.126496 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab2e87-7446-4fa2-8b2b-ca407ec2dfae admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:27.126496 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab2e87-7446-4fa2-8b2b-ca407ec2dfae admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:27.126496 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab2e87-7446-4fa2-8b2b-ca407ec2dfae admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:27.127351 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab2e87-7446-4fa2-8b2b-ca407ec2dfae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:27.127351 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab2e87-7446-4fa2-8b2b-ca407ec2dfae admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:27.160840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1159/2298] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:27.165627 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab2e87-7446-4fa2-8b2b-ca407ec2dfae admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:27.187263 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab2e87-7446-4fa2-8b2b-ca407ec2dfae admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:27.204515 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f7081-9cd9-46f2-801a-e643cf868410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:27.205630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f7081-9cd9-46f2-801a-e643cf868410 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:27.206140 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f7081-9cd9-46f2-801a-e643cf868410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:27.206626 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176f7081-9cd9-46f2-801a-e643cf868410 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:27.207583 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176f7081-9cd9-46f2-801a-e643cf868410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:27.210355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176f7081-9cd9-46f2-801a-e643cf868410 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:27.242929 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1160/2299] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:27.253516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f7081-9cd9-46f2-801a-e643cf868410 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:27.286664 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-4553f6e1-21b8-44d8-b477-88f6c363f7a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba002a25cfaa47cbb738dc1470106c7d'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:27.294505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176f7081-9cd9-46f2-801a-e643cf868410 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:27.306751 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-4553f6e1-21b8-44d8-b477-88f6c363f7a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba002a25cfaa47cbb738dc1470106c7d'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:27.325533 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e7130-9a48-4fd1-85f6-98800ab75878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:27.326548 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e7130-9a48-4fd1-85f6-98800ab75878 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:27.326932 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e7130-9a48-4fd1-85f6-98800ab75878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:27.327311 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e7130-9a48-4fd1-85f6-98800ab75878 admin admin] PATH_INFO: `/v3/projects/1e771a9d362d4a5983afe499a169c27e/users/a3ab6347c284464b8719a0147be8fb0a/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:27.360818 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e7130-9a48-4fd1-85f6-98800ab75878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e771a9d362d4a5983afe499a169c27e, user_id=a3ab6347c284464b8719a0147be8fb0a, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:27.364223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e7130-9a48-4fd1-85f6-98800ab75878 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:27.430465 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-4553f6e1-21b8-44d8-b477-88f6c363f7a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba002a25cfaa47cbb738dc1470106c7d'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:27.431543 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-4553f6e1-21b8-44d8-b477-88f6c363f7a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba002a25cfaa47cbb738dc1470106c7d'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:27.432261 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-4553f6e1-21b8-44d8-b477-88f6c363f7a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba002a25cfaa47cbb738dc1470106c7d'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:27.464438 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1140/2300] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:52:26 2022] DELETE /identity/v3/users/ba002a25cfaa47cbb738dc1470106c7d => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:52:27.469942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af29e7-4ce5-408e-9b55-b534e88a2c32 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:27.470233 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af29e7-4ce5-408e-9b55-b534e88a2c32 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:27.470509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2af29e7-4ce5-408e-9b55-b534e88a2c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:27.502501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1161/2301] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:52:27 2022] PUT /identity/v3/projects/1e771a9d362d4a5983afe499a169c27e/users/a3ab6347c284464b8719a0147be8fb0a/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:27.517567 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65c740-446c-49aa-94dd-eebd56e05c22 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:27.517970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65c740-446c-49aa-94dd-eebd56e05c22 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:27.518355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65c740-446c-49aa-94dd-eebd56e05c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:27.523942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-c2af29e7-4ce5-408e-9b55-b534e88a2c32 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:27.525432 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-c2af29e7-4ce5-408e-9b55-b534e88a2c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:27.606453 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-df65c740-446c-49aa-94dd-eebd56e05c22 None None] MFA Rules not processed for user `a3ab6347c284464b8719a0147be8fb0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:27.608129 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-df65c740-446c-49aa-94dd-eebd56e05c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:27.658340 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1141/2302] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:27.666159 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af29e7-4ce5-408e-9b55-b534e88a2c32 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:27.680056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-c2af29e7-4ce5-408e-9b55-b534e88a2c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:27.794544 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1162/2303] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:27 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:27.795571 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65c740-446c-49aa-94dd-eebd56e05c22 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:27.806505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-df65c740-446c-49aa-94dd-eebd56e05c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:27.815361 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2af29e7-4ce5-408e-9b55-b534e88a2c32 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:27.821560 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e809bb40-1f5a-41f6-9042-53e17ba00d42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:27.824771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809bb40-1f5a-41f6-9042-53e17ba00d42 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:27.825121 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809bb40-1f5a-41f6-9042-53e17ba00d42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:27.826263 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e809bb40-1f5a-41f6-9042-53e17ba00d42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:27.835205 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65c740-446c-49aa-94dd-eebd56e05c22 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:27.847507 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915859da-6e18-4de9-9b21-4eb049b26bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:27.848884 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915859da-6e18-4de9-9b21-4eb049b26bb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:27.848884 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915859da-6e18-4de9-9b21-4eb049b26bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:27.849911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915859da-6e18-4de9-9b21-4eb049b26bb0 admin admin] PATH_INFO: `/v3/users/836484879a494ff98737ae0d3980e886` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:27.852177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e809bb40-1f5a-41f6-9042-53e17ba00d42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:27.852772 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915859da-6e18-4de9-9b21-4eb049b26bb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=836484879a494ff98737ae0d3980e886)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:27.853688 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e809bb40-1f5a-41f6-9042-53e17ba00d42 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:27.855811 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915859da-6e18-4de9-9b21-4eb049b26bb0 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:27.868347 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1142/2304] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:52:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:52:28.145570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-915859da-6e18-4de9-9b21-4eb049b26bb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '836484879a494ff98737ae0d3980e886'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:28.200548 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-915859da-6e18-4de9-9b21-4eb049b26bb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '836484879a494ff98737ae0d3980e886'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:28.312140 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:28.313067 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-915859da-6e18-4de9-9b21-4eb049b26bb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '836484879a494ff98737ae0d3980e886'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:28.313825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-915859da-6e18-4de9-9b21-4eb049b26bb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '836484879a494ff98737ae0d3980e886'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:28.323464 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-915859da-6e18-4de9-9b21-4eb049b26bb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '836484879a494ff98737ae0d3980e886'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:28.345070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1163/2305] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:52:27 2022] DELETE /identity/v3/users/836484879a494ff98737ae0d3980e886 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:28.345425 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b91cd3-4349-4775-ab8f-3747766b79b4 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:28.345425 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b91cd3-4349-4775-ab8f-3747766b79b4 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:28.345425 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b91cd3-4349-4775-ab8f-3747766b79b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:28.397594 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-d8b91cd3-4349-4775-ab8f-3747766b79b4 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:28.409691 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d8b91cd3-4349-4775-ab8f-3747766b79b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:28.452220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:28.564775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1143/2306] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:28.592571 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915859da-6e18-4de9-9b21-4eb049b26bb0 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:28.592571 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-915859da-6e18-4de9-9b21-4eb049b26bb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:28.708868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915859da-6e18-4de9-9b21-4eb049b26bb0 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:28.716468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11947cba-2acf-445e-a5bb-ae1bfa05c317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:28.717169 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11947cba-2acf-445e-a5bb-ae1bfa05c317 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:28.717456 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11947cba-2acf-445e-a5bb-ae1bfa05c317 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:28.717739 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11947cba-2acf-445e-a5bb-ae1bfa05c317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:28.736868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11947cba-2acf-445e-a5bb-ae1bfa05c317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:28.744988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11947cba-2acf-445e-a5bb-ae1bfa05c317 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:28.778681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1164/2307] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:52:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:52:28.895856 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b91cd3-4349-4775-ab8f-3747766b79b4 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:28.932002 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d8b91cd3-4349-4775-ab8f-3747766b79b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:28.932002 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b91cd3-4349-4775-ab8f-3747766b79b4 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:28.946974 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52124f2-b63b-4f98-8bcc-594e50cb5392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:28.952566 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52124f2-b63b-4f98-8bcc-594e50cb5392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:28.954536 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52124f2-b63b-4f98-8bcc-594e50cb5392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:28.954944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52124f2-b63b-4f98-8bcc-594e50cb5392 admin admin] PATH_INFO: `/v3/projects/26987a6990bf4ace9c6674b250955f39` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:28.960045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52124f2-b63b-4f98-8bcc-594e50cb5392 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26987a6990bf4ace9c6674b250955f39)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:28.969340 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52124f2-b63b-4f98-8bcc-594e50cb5392 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:29.249382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c52124f2-b63b-4f98-8bcc-594e50cb5392 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26987a6990bf4ace9c6674b250955f39', 'request_id': 'req-c52124f2-b63b-4f98-8bcc-594e50cb5392'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:29.270264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c52124f2-b63b-4f98-8bcc-594e50cb5392 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26987a6990bf4ace9c6674b250955f39', 'request_id': 'req-c52124f2-b63b-4f98-8bcc-594e50cb5392'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:29.389427 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c52124f2-b63b-4f98-8bcc-594e50cb5392 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26987a6990bf4ace9c6674b250955f39', 'request_id': 'req-c52124f2-b63b-4f98-8bcc-594e50cb5392'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:29.408742 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c52124f2-b63b-4f98-8bcc-594e50cb5392 admin admin] The token cache is being invalidate because project 26987a6990bf4ace9c6674b250955f39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:52:29.409327 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c52124f2-b63b-4f98-8bcc-594e50cb5392 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:29.412648 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c52124f2-b63b-4f98-8bcc-594e50cb5392 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:29.414964 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1144/2308] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:52:28 2022] DELETE /identity/v3/projects/26987a6990bf4ace9c6674b250955f39 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:29.420096 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374a96b-8c6a-4181-aca7-c9ddf690fd97 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:29.421042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374a96b-8c6a-4181-aca7-c9ddf690fd97 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:29.421498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374a96b-8c6a-4181-aca7-c9ddf690fd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:29.470996 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-9374a96b-8c6a-4181-aca7-c9ddf690fd97 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:29.476057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-9374a96b-8c6a-4181-aca7-c9ddf690fd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:29.746332 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1165/2309] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:29.751875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f1e5e-2978-4662-b06e-98d85eb47e95 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:29.752450 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f1e5e-2978-4662-b06e-98d85eb47e95 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:29.752876 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f1e5e-2978-4662-b06e-98d85eb47e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:29.807614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-e36f1e5e-2978-4662-b06e-98d85eb47e95 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:29.809114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-e36f1e5e-2978-4662-b06e-98d85eb47e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:29.842382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1145/2310] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:29.846948 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374a96b-8c6a-4181-aca7-c9ddf690fd97 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:29.860270 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374a96b-8c6a-4181-aca7-c9ddf690fd97 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:29.878146 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a644d-f244-4a1c-8d6c-aa0155dda05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:29.879789 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a644d-f244-4a1c-8d6c-aa0155dda05f admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:29.881165 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a644d-f244-4a1c-8d6c-aa0155dda05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:29.887235 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340a644d-f244-4a1c-8d6c-aa0155dda05f admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:29.888262 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a644d-f244-4a1c-8d6c-aa0155dda05f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:52:29.888738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a644d-f244-4a1c-8d6c-aa0155dda05f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:29.891212 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340a644d-f244-4a1c-8d6c-aa0155dda05f admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:29.920451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1166/2311] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:52:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:52:29.924927 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f1e5e-2978-4662-b06e-98d85eb47e95 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:29.935819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f1e5e-2978-4662-b06e-98d85eb47e95 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:29.943787 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73de069-437c-4e13-a421-150e40a04282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:29.945363 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73de069-437c-4e13-a421-150e40a04282 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:29.945877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73de069-437c-4e13-a421-150e40a04282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:29.946394 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73de069-437c-4e13-a421-150e40a04282 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:29.950105 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73de069-437c-4e13-a421-150e40a04282 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:29.952365 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73de069-437c-4e13-a421-150e40a04282 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:30.444121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a644d-f244-4a1c-8d6c-aa0155dda05f admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:30.445448 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-340a644d-f244-4a1c-8d6c-aa0155dda05f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:30.470784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1146/2312] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:52:29 2022] POST /identity/v3/projects => generated 375 bytes in 543 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:52:30.472667 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73de069-437c-4e13-a421-150e40a04282 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:30.488756 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340a644d-f244-4a1c-8d6c-aa0155dda05f admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:30.627720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:30.647924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73de069-437c-4e13-a421-150e40a04282 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:30.656581 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4b20a-27e3-4a50-956e-a2a5f236ba9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:30.657784 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4b20a-27e3-4a50-956e-a2a5f236ba9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:30.658085 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4b20a-27e3-4a50-956e-a2a5f236ba9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:30.659458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d4b20a-27e3-4a50-956e-a2a5f236ba9e admin admin] PATH_INFO: `/v3/projects/765b703e9d41448cad65d1d1e45418ea` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:30.664126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67cd78-9ff8-471d-b047-f1cd68698824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:30.664602 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4b20a-27e3-4a50-956e-a2a5f236ba9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=765b703e9d41448cad65d1d1e45418ea)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:30.684851 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d4b20a-27e3-4a50-956e-a2a5f236ba9e admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:30.685203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67cd78-9ff8-471d-b047-f1cd68698824 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:30.685203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67cd78-9ff8-471d-b047-f1cd68698824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:30.685203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67cd78-9ff8-471d-b047-f1cd68698824 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:30.685203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e67cd78-9ff8-471d-b047-f1cd68698824 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:30.685203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e67cd78-9ff8-471d-b047-f1cd68698824 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:30.820200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:30.832780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1147/2313] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:52:30 2022] POST /identity/v3/users => generated 288 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:52:30.835887 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67cd78-9ff8-471d-b047-f1cd68698824 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:30.837000 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-27d4b20a-27e3-4a50-956e-a2a5f236ba9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '765b703e9d41448cad65d1d1e45418ea', 'request_id': 'req-27d4b20a-27e3-4a50-956e-a2a5f236ba9e'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:30.849848 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-27d4b20a-27e3-4a50-956e-a2a5f236ba9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '765b703e9d41448cad65d1d1e45418ea', 'request_id': 'req-27d4b20a-27e3-4a50-956e-a2a5f236ba9e'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:30.864487 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-27d4b20a-27e3-4a50-956e-a2a5f236ba9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '765b703e9d41448cad65d1d1e45418ea', 'request_id': 'req-27d4b20a-27e3-4a50-956e-a2a5f236ba9e'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:30.884885 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67cd78-9ff8-471d-b047-f1cd68698824 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:30.966660 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-27d4b20a-27e3-4a50-956e-a2a5f236ba9e admin admin] The token cache is being invalidate because project 765b703e9d41448cad65d1d1e45418ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:52:30.968241 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-27d4b20a-27e3-4a50-956e-a2a5f236ba9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:30.969329 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-27d4b20a-27e3-4a50-956e-a2a5f236ba9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:30.977023 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1167/2314] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:52:30 2022] DELETE /identity/v3/projects/765b703e9d41448cad65d1d1e45418ea => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:52:30.979560 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4b20a-27e3-4a50-956e-a2a5f236ba9e admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:30.986722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-27d4b20a-27e3-4a50-956e-a2a5f236ba9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:31.027764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9deaf4-b18c-4dfc-9483-8306b77d12cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:31.028583 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9deaf4-b18c-4dfc-9483-8306b77d12cb admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:31.029068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9deaf4-b18c-4dfc-9483-8306b77d12cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:31.031411 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9deaf4-b18c-4dfc-9483-8306b77d12cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:31.032362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9deaf4-b18c-4dfc-9483-8306b77d12cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:31.034756 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9deaf4-b18c-4dfc-9483-8306b77d12cb admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:31.051139 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1148/2315] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:30 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:31.059784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9deaf4-b18c-4dfc-9483-8306b77d12cb admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:31.069800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-3f9deaf4-b18c-4dfc-9483-8306b77d12cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:31.112139 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9deaf4-b18c-4dfc-9483-8306b77d12cb admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:31.129581 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29da08-9fe0-40f6-bad4-41b882fc28bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:31.130478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29da08-9fe0-40f6-bad4-41b882fc28bc admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:31.130848 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29da08-9fe0-40f6-bad4-41b882fc28bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:31.132652 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29da08-9fe0-40f6-bad4-41b882fc28bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:31.133707 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29da08-9fe0-40f6-bad4-41b882fc28bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:31.141396 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae29da08-9fe0-40f6-bad4-41b882fc28bc admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:31.172849 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1149/2316] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:31 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:31.176821 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29da08-9fe0-40f6-bad4-41b882fc28bc admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:31.207375 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d4b20a-27e3-4a50-956e-a2a5f236ba9e admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:31.217053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29da08-9fe0-40f6-bad4-41b882fc28bc admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:31.227414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358074f-03ce-4855-b88e-422d41cd56ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:31.238634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358074f-03ce-4855-b88e-422d41cd56ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:31.238634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358074f-03ce-4855-b88e-422d41cd56ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:31.238634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9358074f-03ce-4855-b88e-422d41cd56ee admin admin] PATH_INFO: `/v3/users/27e27a4d52494b0cbd30c33c9405b937` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:31.238926 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761ba89-8269-469f-8acc-549141074930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:31.238926 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761ba89-8269-469f-8acc-549141074930 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:31.238926 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761ba89-8269-469f-8acc-549141074930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:31.239180 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358074f-03ce-4855-b88e-422d41cd56ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=27e27a4d52494b0cbd30c33c9405b937)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:31.239265 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761ba89-8269-469f-8acc-549141074930 admin admin] PATH_INFO: `/v3/projects/96ab6f56bbb34547a1fa2548159e7579/users/f7870acb5f8b447e8ebb86eaa2ecfcfa/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:31.242097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9358074f-03ce-4855-b88e-422d41cd56ee admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:31.281848 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761ba89-8269-469f-8acc-549141074930 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96ab6f56bbb34547a1fa2548159e7579, user_id=f7870acb5f8b447e8ebb86eaa2ecfcfa, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:31.290808 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761ba89-8269-469f-8acc-549141074930 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:31.388679 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1150/2317] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:52:31 2022] PUT /identity/v3/projects/96ab6f56bbb34547a1fa2548159e7579/users/f7870acb5f8b447e8ebb86eaa2ecfcfa/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:31.410585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af088a-76a8-44a9-9b61-2804cf7409e6 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:31.410585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af088a-76a8-44a9-9b61-2804cf7409e6 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:31.410585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af088a-76a8-44a9-9b61-2804cf7409e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:31.485410 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-22af088a-76a8-44a9-9b61-2804cf7409e6 None None] MFA Rules not processed for user `f7870acb5f8b447e8ebb86eaa2ecfcfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:31.486706 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-22af088a-76a8-44a9-9b61-2804cf7409e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:31.533549 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-9358074f-03ce-4855-b88e-422d41cd56ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27e27a4d52494b0cbd30c33c9405b937'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:31.585307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-9358074f-03ce-4855-b88e-422d41cd56ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27e27a4d52494b0cbd30c33c9405b937'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:31.654477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:31.687513 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-9358074f-03ce-4855-b88e-422d41cd56ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27e27a4d52494b0cbd30c33c9405b937'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:31.695237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-9358074f-03ce-4855-b88e-422d41cd56ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27e27a4d52494b0cbd30c33c9405b937'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:31.705795 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-9358074f-03ce-4855-b88e-422d41cd56ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27e27a4d52494b0cbd30c33c9405b937'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:31.728623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1168/2318] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:52:30 2022] DELETE /identity/v3/users/27e27a4d52494b0cbd30c33c9405b937 => generated 0 bytes in 748 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:31.768888 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1151/2319] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:52:31.776812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358074f-03ce-4855-b88e-422d41cd56ee admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:31.778944 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-9358074f-03ce-4855-b88e-422d41cd56ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:31.790410 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:31.866529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9358074f-03ce-4855-b88e-422d41cd56ee admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:31.875150 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0867c37-07c8-40b7-98e8-a70e6f48cb2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:31.876485 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0867c37-07c8-40b7-98e8-a70e6f48cb2e service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:31.877072 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0867c37-07c8-40b7-98e8-a70e6f48cb2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:31.877701 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0867c37-07c8-40b7-98e8-a70e6f48cb2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:32.027451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0867c37-07c8-40b7-98e8-a70e6f48cb2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:32.028916 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0867c37-07c8-40b7-98e8-a70e6f48cb2e service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:32.048693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1169/2320] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:52:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:52:32.098569 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:32.380741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:32.628232 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af088a-76a8-44a9-9b61-2804cf7409e6 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:32.629568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-22af088a-76a8-44a9-9b61-2804cf7409e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:32.688094 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:32.841474 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af088a-76a8-44a9-9b61-2804cf7409e6 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:33.041840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:33.073465 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012d40e4-ef7c-45e8-815e-9a49e8d18e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:33.074748 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d40e4-ef7c-45e8-815e-9a49e8d18e60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:33.075112 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d40e4-ef7c-45e8-815e-9a49e8d18e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:33.075512 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012d40e4-ef7c-45e8-815e-9a49e8d18e60 admin admin] PATH_INFO: `/v3/projects/174102b431064258965e637360d32c86` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:33.077953 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d40e4-ef7c-45e8-815e-9a49e8d18e60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=174102b431064258965e637360d32c86)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:33.081672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012d40e4-ef7c-45e8-815e-9a49e8d18e60 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:33.325243 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-012d40e4-ef7c-45e8-815e-9a49e8d18e60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '174102b431064258965e637360d32c86', 'request_id': 'req-012d40e4-ef7c-45e8-815e-9a49e8d18e60'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:33.340847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-012d40e4-ef7c-45e8-815e-9a49e8d18e60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '174102b431064258965e637360d32c86', 'request_id': 'req-012d40e4-ef7c-45e8-815e-9a49e8d18e60'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:33.442097 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-012d40e4-ef7c-45e8-815e-9a49e8d18e60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '174102b431064258965e637360d32c86', 'request_id': 'req-012d40e4-ef7c-45e8-815e-9a49e8d18e60'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:33.466949 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-012d40e4-ef7c-45e8-815e-9a49e8d18e60 admin admin] The token cache is being invalidate because project 174102b431064258965e637360d32c86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:52:33.467622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-012d40e4-ef7c-45e8-815e-9a49e8d18e60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:33.470229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-012d40e4-ef7c-45e8-815e-9a49e8d18e60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:52:33.476016 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1152/2321] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:52:32 2022] DELETE /identity/v3/projects/174102b431064258965e637360d32c86 => generated 0 bytes in 848 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:33.480792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d851b-3fef-46f1-8c3b-708a4848492a None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:33.481113 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d851b-3fef-46f1-8c3b-708a4848492a None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:33.481586 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d851b-3fef-46f1-8c3b-708a4848492a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:33.543824 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-836d851b-3fef-46f1-8c3b-708a4848492a None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:33.551722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-836d851b-3fef-46f1-8c3b-708a4848492a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:33.609348 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:33.743861 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1170/2322] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:33.753150 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b45b0e-22dc-46a5-9cab-c940329702bc None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:33.753606 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b45b0e-22dc-46a5-9cab-c940329702bc None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:33.754868 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b45b0e-22dc-46a5-9cab-c940329702bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:33.782674 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-05b45b0e-22dc-46a5-9cab-c940329702bc None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:33.784052 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-05b45b0e-22dc-46a5-9cab-c940329702bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:33.811676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1153/2323] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:33.816039 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d851b-3fef-46f1-8c3b-708a4848492a None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:33.830223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d851b-3fef-46f1-8c3b-708a4848492a None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:33.838829 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae1f621-2495-43dc-9189-70ee385d75a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:33.840539 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1f621-2495-43dc-9189-70ee385d75a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:33.840879 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1f621-2495-43dc-9189-70ee385d75a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:33.841415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae1f621-2495-43dc-9189-70ee385d75a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:33.842314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae1f621-2495-43dc-9189-70ee385d75a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:52:33.842635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae1f621-2495-43dc-9189-70ee385d75a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:33.844606 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae1f621-2495-43dc-9189-70ee385d75a4 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:33.864651 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b45b0e-22dc-46a5-9cab-c940329702bc None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:33.875422 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1171/2324] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:52:33.879660 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b45b0e-22dc-46a5-9cab-c940329702bc None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:33.886945 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03c4f0-1cd0-4c47-aee7-7475e127cbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:33.887639 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03c4f0-1cd0-4c47-aee7-7475e127cbda admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:33.887926 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03c4f0-1cd0-4c47-aee7-7475e127cbda admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:33.888213 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03c4f0-1cd0-4c47-aee7-7475e127cbda admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:33.888915 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de03c4f0-1cd0-4c47-aee7-7475e127cbda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:33.892773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de03c4f0-1cd0-4c47-aee7-7475e127cbda admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:34.033308 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae1f621-2495-43dc-9189-70ee385d75a4 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:34.033862 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1154/2325] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:52:33 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:52:34.058695 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae1f621-2495-43dc-9189-70ee385d75a4 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:34.216974 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64774d5-a376-4714-a447-88ceb25f64fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:34.218091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64774d5-a376-4714-a447-88ceb25f64fe admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:34.218723 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64774d5-a376-4714-a447-88ceb25f64fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:34.219127 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64774d5-a376-4714-a447-88ceb25f64fe admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:34.220019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64774d5-a376-4714-a447-88ceb25f64fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:34.222265 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64774d5-a376-4714-a447-88ceb25f64fe admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:34.350782 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:34.362586 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1172/2326] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:52:34 2022] POST /identity/v3/users => generated 289 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:52:34.365925 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03c4f0-1cd0-4c47-aee7-7475e127cbda admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:34.386539 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03c4f0-1cd0-4c47-aee7-7475e127cbda admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:34.401695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329fe777-3875-4b77-aeca-5f79cd2b5d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:34.405751 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fe777-3875-4b77-aeca-5f79cd2b5d3b admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:34.406173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fe777-3875-4b77-aeca-5f79cd2b5d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:34.407358 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329fe777-3875-4b77-aeca-5f79cd2b5d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:34.409487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fe777-3875-4b77-aeca-5f79cd2b5d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:34.415411 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329fe777-3875-4b77-aeca-5f79cd2b5d3b admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:34.440578 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64774d5-a376-4714-a447-88ceb25f64fe admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:34.445841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1155/2327] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:34.445841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87410563-835c-40e3-9da9-e1520e543db1 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:34.445841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87410563-835c-40e3-9da9-e1520e543db1 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:34.445841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87410563-835c-40e3-9da9-e1520e543db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:34.465110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64774d5-a376-4714-a447-88ceb25f64fe admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:34.465770 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-87410563-835c-40e3-9da9-e1520e543db1 None None] MFA Rules not processed for user `f7870acb5f8b447e8ebb86eaa2ecfcfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:34.467353 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-87410563-835c-40e3-9da9-e1520e543db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:34.493011 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b439ad6-c662-47a6-99ad-3de8f2603ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:34.493870 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b439ad6-c662-47a6-99ad-3de8f2603ff6 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:34.494133 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b439ad6-c662-47a6-99ad-3de8f2603ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:34.494426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b439ad6-c662-47a6-99ad-3de8f2603ff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:34.495157 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b439ad6-c662-47a6-99ad-3de8f2603ff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:34.496834 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b439ad6-c662-47a6-99ad-3de8f2603ff6 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:34.547880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1173/2328] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:52:34.549003 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b439ad6-c662-47a6-99ad-3de8f2603ff6 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:34.568803 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1156/2329] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:34.580203 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b439ad6-c662-47a6-99ad-3de8f2603ff6 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:34.595851 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0e4dd-4d44-4af9-9ee8-348aca8bb73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:34.595851 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0e4dd-4d44-4af9-9ee8-348aca8bb73b admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:34.596435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0e4dd-4d44-4af9-9ee8-348aca8bb73b admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:34.596903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0e4dd-4d44-4af9-9ee8-348aca8bb73b admin admin] PATH_INFO: `/v3/projects/cab9310244124b71bc814d1a0ed19920/users/46b07bef66a84049957ab82527c54427/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:34.613355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:34.631207 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0e4dd-4d44-4af9-9ee8-348aca8bb73b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cab9310244124b71bc814d1a0ed19920, user_id=46b07bef66a84049957ab82527c54427, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:34.634582 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0e4dd-4d44-4af9-9ee8-348aca8bb73b admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:34.743724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1174/2330] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:52:34 2022] PUT /identity/v3/projects/cab9310244124b71bc814d1a0ed19920/users/46b07bef66a84049957ab82527c54427/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:52:34.746239 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4f2ed-1556-42c0-a316-4b3c99d4542c None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:34.746555 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4f2ed-1556-42c0-a316-4b3c99d4542c None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:34.746833 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c4f2ed-1556-42c0-a316-4b3c99d4542c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:34.800280 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-49c4f2ed-1556-42c0-a316-4b3c99d4542c None None] MFA Rules not processed for user `46b07bef66a84049957ab82527c54427`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:34.807004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-49c4f2ed-1556-42c0-a316-4b3c99d4542c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:34.844699 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:34.917522 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1157/2331] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:34.934112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0e4dd-4d44-4af9-9ee8-348aca8bb73b admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:34.935940 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-4fd0e4dd-4d44-4af9-9ee8-348aca8bb73b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:35.082205 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:35.189314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0e4dd-4d44-4af9-9ee8-348aca8bb73b admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:35.199303 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0576dd70-2308-4a21-b040-81430cb303a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:35.201874 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576dd70-2308-4a21-b040-81430cb303a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:35.202183 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576dd70-2308-4a21-b040-81430cb303a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:35.202465 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576dd70-2308-4a21-b040-81430cb303a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:35.227632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0576dd70-2308-4a21-b040-81430cb303a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:35.229741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0576dd70-2308-4a21-b040-81430cb303a3 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:35.237898 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:35.265406 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1175/2332] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:52:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:52:35.784549 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:37.922202 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4f2ed-1556-42c0-a316-4b3c99d4542c None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:37.929757 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-49c4f2ed-1556-42c0-a316-4b3c99d4542c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:38.045914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c4f2ed-1556-42c0-a316-4b3c99d4542c None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:38.050045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5465ce-dd65-4af2-b061-9e6581988a76 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1347c0bd4ada4139ad194d423958be6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:38.050825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5465ce-dd65-4af2-b061-9e6581988a76 service glance] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:38.057058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5465ce-dd65-4af2-b061-9e6581988a76 service glance] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:38.064036 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5465ce-dd65-4af2-b061-9e6581988a76 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:38.066875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-ee5465ce-dd65-4af2-b061-9e6581988a76 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:38.156544 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5465ce-dd65-4af2-b061-9e6581988a76 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:38.158318 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5465ce-dd65-4af2-b061-9e6581988a76 service glance] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:38.172795 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1158/2333] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:52:37 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:52:45.875428 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:48.165137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:52:52.841279 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df679c-70f3-4c1e-9222-def84376144d None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:52.843451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df679c-70f3-4c1e-9222-def84376144d None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:52.843451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df679c-70f3-4c1e-9222-def84376144d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:52.860593 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-40df679c-70f3-4c1e-9222-def84376144d None None] MFA Rules not processed for user `83d3494760b84bc1aad951bba11caea3`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:52.861837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-40df679c-70f3-4c1e-9222-def84376144d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:52.922964 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1176/2334] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:52:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:53.827180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc3ebc-40bd-49e1-bb1e-9010a20ffcfb None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:53.835004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc3ebc-40bd-49e1-bb1e-9010a20ffcfb None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:53.835004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fc3ebc-40bd-49e1-bb1e-9010a20ffcfb None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:53.835004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1159/2335] 10.222.0.13 () {54 vars in 921 bytes} [Fri May 27 19:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:52:53.835430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d78a31-a039-4d20-b02c-0fe4e1459435 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:53.835765 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d78a31-a039-4d20-b02c-0fe4e1459435 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:53.836049 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d78a31-a039-4d20-b02c-0fe4e1459435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:53.906578 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-21d78a31-a039-4d20-b02c-0fe4e1459435 None None] MFA Rules not processed for user `7542e1fd160e4b2c9d25accf19aad19f`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:53.907990 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-21d78a31-a039-4d20-b02c-0fe4e1459435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:53.991357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1177/2336] 10.222.0.13 () {58 vars in 1018 bytes} [Fri May 27 19:52:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:54.003424 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc3ebc-40bd-49e1-bb1e-9010a20ffcfb None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:54.006512 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-68fc3ebc-40bd-49e1-bb1e-9010a20ffcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:54.039796 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fc3ebc-40bd-49e1-bb1e-9010a20ffcfb None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:54.043916 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2f3800-c9d0-4a54-a8f3-7bd1b8744822 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1347c0bd4ada4139ad194d423958be6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:54.044614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f3800-c9d0-4a54-a8f3-7bd1b8744822 service glance] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:54.044910 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f3800-c9d0-4a54-a8f3-7bd1b8744822 service glance] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:54.045228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2f3800-c9d0-4a54-a8f3-7bd1b8744822 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:54.064007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2f3800-c9d0-4a54-a8f3-7bd1b8744822 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:54.065498 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2f3800-c9d0-4a54-a8f3-7bd1b8744822 service glance] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:54.078583 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1160/2337] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:52:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:52:54.293503 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab97580-33f9-4299-89b3-fcec56f59670 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:54.293873 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab97580-33f9-4299-89b3-fcec56f59670 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:54.294159 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab97580-33f9-4299-89b3-fcec56f59670 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:54.295679 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1178/2338] 10.222.0.13 () {54 vars in 921 bytes} [Fri May 27 19:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:52:54.300909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fc552-b4da-4f8c-8004-07ae15d7b2d0 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:54.301490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fc552-b4da-4f8c-8004-07ae15d7b2d0 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:54.301817 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fc552-b4da-4f8c-8004-07ae15d7b2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:54.320782 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-de7fc552-b4da-4f8c-8004-07ae15d7b2d0 None None] MFA Rules not processed for user `dd34f15400c4476bbb2161dbe1b8b2b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:54.322008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-de7fc552-b4da-4f8c-8004-07ae15d7b2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:54.375740 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1161/2339] 10.222.0.13 () {58 vars in 1018 bytes} [Fri May 27 19:52:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:54.387665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab97580-33f9-4299-89b3-fcec56f59670 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:54.389365 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-cab97580-33f9-4299-89b3-fcec56f59670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:54.402962 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab97580-33f9-4299-89b3-fcec56f59670 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:54.406560 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6204bf-5216-46dd-8d16-80e2d756d646 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:54.407191 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6204bf-5216-46dd-8d16-80e2d756d646 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:54.407476 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6204bf-5216-46dd-8d16-80e2d756d646 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:54.409072 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6204bf-5216-46dd-8d16-80e2d756d646 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:54.415098 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6204bf-5216-46dd-8d16-80e2d756d646 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:54.416601 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6204bf-5216-46dd-8d16-80e2d756d646 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:54.425564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1179/2340] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:52:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:52:54.518405 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40689246-6e56-4f6f-a729-25783abad5a3 None None] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:54.518834 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40689246-6e56-4f6f-a729-25783abad5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:54.519191 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40689246-6e56-4f6f-a729-25783abad5a3 None None] PATH_INFO: `/` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:54.521039 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1162/2341] 10.222.0.13 () {54 vars in 921 bytes} [Fri May 27 19:52:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:52:54.527747 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60aa57-b372-4482-b8f4-1acbc6261c7e None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:54.528231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60aa57-b372-4482-b8f4-1acbc6261c7e None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:54.528519 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef60aa57-b372-4482-b8f4-1acbc6261c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:54.548155 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-ef60aa57-b372-4482-b8f4-1acbc6261c7e None None] MFA Rules not processed for user `7542e1fd160e4b2c9d25accf19aad19f`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:52:54.549504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ef60aa57-b372-4482-b8f4-1acbc6261c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:54.560217 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1180/2342] 10.222.0.13 () {58 vars in 1018 bytes} [Fri May 27 19:52:54 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:52:54.571842 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40689246-6e56-4f6f-a729-25783abad5a3 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:52:54.573627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-40689246-6e56-4f6f-a729-25783abad5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:52:54.587224 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40689246-6e56-4f6f-a729-25783abad5a3 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:52:54.590901 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fdd6db-31fb-46fd-8187-d1082dbcf3cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:52:54.591544 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdd6db-31fb-46fd-8187-d1082dbcf3cd service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:52:54.591991 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdd6db-31fb-46fd-8187-d1082dbcf3cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:52:54.592278 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdd6db-31fb-46fd-8187-d1082dbcf3cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:52:54.602699 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fdd6db-31fb-46fd-8187-d1082dbcf3cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:52:54.604169 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fdd6db-31fb-46fd-8187-d1082dbcf3cd service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:52:54.618737 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1163/2343] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:52:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:53:01.545209 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60aa57-b372-4482-b8f4-1acbc6261c7e None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:01.682221 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:53:01.707082 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef60aa57-b372-4482-b8f4-1acbc6261c7e None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:01.712740 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149bc896-9ded-4639-accd-05bd21d8604e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:01.713789 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bc896-9ded-4639-accd-05bd21d8604e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:01.714077 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bc896-9ded-4639-accd-05bd21d8604e admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:01.714360 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bc896-9ded-4639-accd-05bd21d8604e admin admin] PATH_INFO: `/v3/users/46b07bef66a84049957ab82527c54427` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:01.715913 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149bc896-9ded-4639-accd-05bd21d8604e admin admin] RBAC: Authorizing `identity:delete_user(user_id=46b07bef66a84049957ab82527c54427)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:01.718560 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149bc896-9ded-4639-accd-05bd21d8604e admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:01.918011 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-149bc896-9ded-4639-accd-05bd21d8604e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46b07bef66a84049957ab82527c54427'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:01.944260 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-149bc896-9ded-4639-accd-05bd21d8604e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46b07bef66a84049957ab82527c54427'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:02.041784 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-149bc896-9ded-4639-accd-05bd21d8604e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46b07bef66a84049957ab82527c54427'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:02.042846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-149bc896-9ded-4639-accd-05bd21d8604e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46b07bef66a84049957ab82527c54427'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:02.055287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-149bc896-9ded-4639-accd-05bd21d8604e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46b07bef66a84049957ab82527c54427'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:02.062694 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1181/2344] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:53:01 2022] DELETE /identity/v3/users/46b07bef66a84049957ab82527c54427 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:53:02.067576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdaf1c-ac85-4cbf-ae6f-2ad35dacdd94 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:02.068746 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdaf1c-ac85-4cbf-ae6f-2ad35dacdd94 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:02.069039 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afdaf1c-ac85-4cbf-ae6f-2ad35dacdd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:02.089497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-0afdaf1c-ac85-4cbf-ae6f-2ad35dacdd94 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:53:02.090779 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-0afdaf1c-ac85-4cbf-ae6f-2ad35dacdd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:02.158337 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1164/2345] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:53:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:53:02.169377 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149bc896-9ded-4639-accd-05bd21d8604e admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:02.171592 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-149bc896-9ded-4639-accd-05bd21d8604e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:02.249289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149bc896-9ded-4639-accd-05bd21d8604e admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:02.253142 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb1ca3-3749-49bf-bb7b-d1384865f7c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:02.253655 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb1ca3-3749-49bf-bb7b-d1384865f7c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:02.253943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb1ca3-3749-49bf-bb7b-d1384865f7c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:02.255912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb1ca3-3749-49bf-bb7b-d1384865f7c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:02.264532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb1ca3-3749-49bf-bb7b-d1384865f7c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:02.266019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb1ca3-3749-49bf-bb7b-d1384865f7c6 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:02.276517 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1182/2346] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:53:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:53:02.870889 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afdaf1c-ac85-4cbf-ae6f-2ad35dacdd94 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:02.872795 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-0afdaf1c-ac85-4cbf-ae6f-2ad35dacdd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:02.886064 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afdaf1c-ac85-4cbf-ae6f-2ad35dacdd94 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:02.891306 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fc81c2-a582-4c10-a693-806a17f15e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:02.893047 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc81c2-a582-4c10-a693-806a17f15e4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:02.893047 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc81c2-a582-4c10-a693-806a17f15e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:02.893047 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fc81c2-a582-4c10-a693-806a17f15e4c admin admin] PATH_INFO: `/v3/projects/cab9310244124b71bc814d1a0ed19920` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:02.893894 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc81c2-a582-4c10-a693-806a17f15e4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cab9310244124b71bc814d1a0ed19920)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:02.896512 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fc81c2-a582-4c10-a693-806a17f15e4c admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:03.057325 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-63fc81c2-a582-4c10-a693-806a17f15e4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cab9310244124b71bc814d1a0ed19920', 'request_id': 'req-63fc81c2-a582-4c10-a693-806a17f15e4c'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:03.063168 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-63fc81c2-a582-4c10-a693-806a17f15e4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cab9310244124b71bc814d1a0ed19920', 'request_id': 'req-63fc81c2-a582-4c10-a693-806a17f15e4c'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:03.143551 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-63fc81c2-a582-4c10-a693-806a17f15e4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cab9310244124b71bc814d1a0ed19920', 'request_id': 'req-63fc81c2-a582-4c10-a693-806a17f15e4c'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:03.157589 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-63fc81c2-a582-4c10-a693-806a17f15e4c admin admin] The token cache is being invalidate because project cab9310244124b71bc814d1a0ed19920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:53:03.157883 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-63fc81c2-a582-4c10-a693-806a17f15e4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:03.158579 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-63fc81c2-a582-4c10-a693-806a17f15e4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:03.160120 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1165/2347] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:53:02 2022] DELETE /identity/v3/projects/cab9310244124b71bc814d1a0ed19920 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:53:03.165219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aace840-d7bc-4b71-92be-5fbdf9d679e5 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:03.165684 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aace840-d7bc-4b71-92be-5fbdf9d679e5 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:03.165780 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aace840-d7bc-4b71-92be-5fbdf9d679e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:03.184507 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-5aace840-d7bc-4b71-92be-5fbdf9d679e5 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:53:03.185759 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-5aace840-d7bc-4b71-92be-5fbdf9d679e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:03.247534 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1183/2348] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:53:03.255219 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af505291-ef60-4b82-9309-7909baf954ea None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:03.255676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af505291-ef60-4b82-9309-7909baf954ea None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:03.255987 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af505291-ef60-4b82-9309-7909baf954ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:03.268386 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:53:03.274533 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-af505291-ef60-4b82-9309-7909baf954ea None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:53:03.275722 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-af505291-ef60-4b82-9309-7909baf954ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:03.285086 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1166/2349] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:53:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:53:03.289904 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aace840-d7bc-4b71-92be-5fbdf9d679e5 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:03.299703 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aace840-d7bc-4b71-92be-5fbdf9d679e5 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:03.305224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59304d9b-83d8-4e37-8625-97c4d3e380ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:03.305882 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59304d9b-83d8-4e37-8625-97c4d3e380ac admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:03.306169 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59304d9b-83d8-4e37-8625-97c4d3e380ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:03.306448 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59304d9b-83d8-4e37-8625-97c4d3e380ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:03.307175 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59304d9b-83d8-4e37-8625-97c4d3e380ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:53:03.307484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59304d9b-83d8-4e37-8625-97c4d3e380ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:03.309117 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59304d9b-83d8-4e37-8625-97c4d3e380ac admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:03.310614 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:53:03.322306 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1184/2350] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:53:03.326615 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af505291-ef60-4b82-9309-7909baf954ea None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:03.336134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af505291-ef60-4b82-9309-7909baf954ea None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:03.341475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9638281-418c-4aeb-a824-2dadc26be7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:03.342139 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9638281-418c-4aeb-a824-2dadc26be7eb admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:03.342684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9638281-418c-4aeb-a824-2dadc26be7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:03.342967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9638281-418c-4aeb-a824-2dadc26be7eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:03.344340 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9638281-418c-4aeb-a824-2dadc26be7eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:03.345707 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9638281-418c-4aeb-a824-2dadc26be7eb admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:03.418387 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1167/2351] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:53:03 2022] POST /identity/v3/projects => generated 371 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:53:03.424434 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59304d9b-83d8-4e37-8625-97c4d3e380ac admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:03.435310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59304d9b-83d8-4e37-8625-97c4d3e380ac admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:03.493075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f774a84-5136-4d0b-9e84-1ea6439f788a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:03.494045 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f774a84-5136-4d0b-9e84-1ea6439f788a admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:03.494386 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f774a84-5136-4d0b-9e84-1ea6439f788a admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:03.494969 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f774a84-5136-4d0b-9e84-1ea6439f788a admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:03.495402 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f774a84-5136-4d0b-9e84-1ea6439f788a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:03.497367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f774a84-5136-4d0b-9e84-1ea6439f788a admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:03.625779 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1185/2352] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:53:03 2022] POST /identity/v3/users => generated 286 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:53:03.627458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9638281-418c-4aeb-a824-2dadc26be7eb admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:03.639660 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9638281-418c-4aeb-a824-2dadc26be7eb admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:03.644603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20342f-3264-4d6b-bcff-4ac4ebffc1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:03.645312 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20342f-3264-4d6b-bcff-4ac4ebffc1ae admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:03.645601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20342f-3264-4d6b-bcff-4ac4ebffc1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:03.645942 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf20342f-3264-4d6b-bcff-4ac4ebffc1ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:03.646800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20342f-3264-4d6b-bcff-4ac4ebffc1ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:03.648434 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf20342f-3264-4d6b-bcff-4ac4ebffc1ae admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:03.660986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1168/2353] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:53:03.663572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f774a84-5136-4d0b-9e84-1ea6439f788a admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:03.673686 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f774a84-5136-4d0b-9e84-1ea6439f788a admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:03.679618 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da14a83-f8f4-4307-a42a-0b7a8c130296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:03.680294 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da14a83-f8f4-4307-a42a-0b7a8c130296 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:03.680582 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da14a83-f8f4-4307-a42a-0b7a8c130296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:03.680859 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da14a83-f8f4-4307-a42a-0b7a8c130296 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:03.681629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da14a83-f8f4-4307-a42a-0b7a8c130296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:03.683239 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da14a83-f8f4-4307-a42a-0b7a8c130296 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:03.693832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1186/2354] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:53:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:53:03.698039 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20342f-3264-4d6b-bcff-4ac4ebffc1ae admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:03.708074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf20342f-3264-4d6b-bcff-4ac4ebffc1ae admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:03.713567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6172f99-9f23-41e0-a15d-00fb9c2d2156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:03.714475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6172f99-9f23-41e0-a15d-00fb9c2d2156 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:03.714765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6172f99-9f23-41e0-a15d-00fb9c2d2156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:03.715044 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6172f99-9f23-41e0-a15d-00fb9c2d2156 admin admin] PATH_INFO: `/v3/projects/130a0d7717ff460ea894123e5c6cc428/users/6c0be04f176b4529828556fd74507354/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:03.731566 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6172f99-9f23-41e0-a15d-00fb9c2d2156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=130a0d7717ff460ea894123e5c6cc428, user_id=6c0be04f176b4529828556fd74507354, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:03.734287 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6172f99-9f23-41e0-a15d-00fb9c2d2156 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:03.801046 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1169/2355] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:53:03 2022] PUT /identity/v3/projects/130a0d7717ff460ea894123e5c6cc428/users/6c0be04f176b4529828556fd74507354/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:53:03.811308 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba061b-2b81-4f10-81e6-784422e08392 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:03.811607 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba061b-2b81-4f10-81e6-784422e08392 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:03.811897 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bba061b-2b81-4f10-81e6-784422e08392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:03.840276 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-4bba061b-2b81-4f10-81e6-784422e08392 None None] MFA Rules not processed for user `6c0be04f176b4529828556fd74507354`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:53:03.841528 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-4bba061b-2b81-4f10-81e6-784422e08392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:03.908147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1187/2356] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:53:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:53:03.920877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6172f99-9f23-41e0-a15d-00fb9c2d2156 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:03.923597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d6172f99-9f23-41e0-a15d-00fb9c2d2156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:03.989253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6172f99-9f23-41e0-a15d-00fb9c2d2156 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:03.993665 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761d4ed-6b6a-4d92-a666-3b2340968171 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1347c0bd4ada4139ad194d423958be6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:03.994324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761d4ed-6b6a-4d92-a666-3b2340968171 service glance] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:03.994614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761d4ed-6b6a-4d92-a666-3b2340968171 service glance] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:03.994897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0761d4ed-6b6a-4d92-a666-3b2340968171 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:04.004386 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761d4ed-6b6a-4d92-a666-3b2340968171 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:04.006985 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0761d4ed-6b6a-4d92-a666-3b2340968171 service glance] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:04.016765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1170/2357] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:53:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:53:05.637481 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac30da9-986e-4a65-b0dd-b85d35b47568 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:05.640872 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac30da9-986e-4a65-b0dd-b85d35b47568 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:05.640872 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac30da9-986e-4a65-b0dd-b85d35b47568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:05.694753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-dac30da9-986e-4a65-b0dd-b85d35b47568 None None] MFA Rules not processed for user `c700b0adb9f54947a88c1baa86accb9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:53:05.696099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-dac30da9-986e-4a65-b0dd-b85d35b47568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:05.731990 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:53:05.765605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1188/2358] 10.222.0.13 () {58 vars in 1009 bytes} [Fri May 27 19:53:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:53:05.797483 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761d4ed-6b6a-4d92-a666-3b2340968171 service glance] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:05.797483 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-0761d4ed-6b6a-4d92-a666-3b2340968171 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:05.932309 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0761d4ed-6b6a-4d92-a666-3b2340968171 service glance] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:05.937627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb3a06-7076-42d8-bb42-f2bc42029d75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37bc6d0abb454de19e1ab2f46bf4a593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:05.938284 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb3a06-7076-42d8-bb42-f2bc42029d75 service swift] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:05.938574 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb3a06-7076-42d8-bb42-f2bc42029d75 service swift] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:05.938856 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb3a06-7076-42d8-bb42-f2bc42029d75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:05.948436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb3a06-7076-42d8-bb42-f2bc42029d75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:05.950952 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb3a06-7076-42d8-bb42-f2bc42029d75 service swift] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:05.963152 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1171/2359] 10.222.0.13 () {60 vars in 1384 bytes} [Fri May 27 19:53:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 19:53:15.909311 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=12 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:53:15.958836 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=91,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:53:21.595055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac30da9-986e-4a65-b0dd-b85d35b47568 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:21.600907 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-dac30da9-986e-4a65-b0dd-b85d35b47568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:21.664101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac30da9-986e-4a65-b0dd-b85d35b47568 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:21.668576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab3f81-5a79-4cc6-9020-1abf25e693ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:21.669482 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab3f81-5a79-4cc6-9020-1abf25e693ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:21.669766 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab3f81-5a79-4cc6-9020-1abf25e693ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:21.670068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab3f81-5a79-4cc6-9020-1abf25e693ce admin admin] PATH_INFO: `/v3/users/a3ab6347c284464b8719a0147be8fb0a` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:21.671267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab3f81-5a79-4cc6-9020-1abf25e693ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3ab6347c284464b8719a0147be8fb0a)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:21.673245 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab3f81-5a79-4cc6-9020-1abf25e693ce admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:21.830678 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-2bab3f81-5a79-4cc6-9020-1abf25e693ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3ab6347c284464b8719a0147be8fb0a'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:21.856194 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-2bab3f81-5a79-4cc6-9020-1abf25e693ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3ab6347c284464b8719a0147be8fb0a'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:21.936697 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-2bab3f81-5a79-4cc6-9020-1abf25e693ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3ab6347c284464b8719a0147be8fb0a'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:21.937335 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-2bab3f81-5a79-4cc6-9020-1abf25e693ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3ab6347c284464b8719a0147be8fb0a'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:21.945436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-2bab3f81-5a79-4cc6-9020-1abf25e693ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3ab6347c284464b8719a0147be8fb0a'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:21.951414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1189/2360] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:53:21 2022] DELETE /identity/v3/users/a3ab6347c284464b8719a0147be8fb0a => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:53:21.955961 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d29c6-aad4-4243-b080-c594c83a9f78 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:21.956452 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d29c6-aad4-4243-b080-c594c83a9f78 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:21.956710 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920d29c6-aad4-4243-b080-c594c83a9f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:21.976147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-920d29c6-aad4-4243-b080-c594c83a9f78 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:53:21.977384 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-920d29c6-aad4-4243-b080-c594c83a9f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:22.037538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1172/2361] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:53:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:53:22.047095 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab3f81-5a79-4cc6-9020-1abf25e693ce admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:22.048422 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-2bab3f81-5a79-4cc6-9020-1abf25e693ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:22.104889 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab3f81-5a79-4cc6-9020-1abf25e693ce admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:22.108473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21054cbe-a37f-4175-9e61-c8035decf680 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:22.109148 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21054cbe-a37f-4175-9e61-c8035decf680 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:22.109434 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21054cbe-a37f-4175-9e61-c8035decf680 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:22.109714 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21054cbe-a37f-4175-9e61-c8035decf680 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:22.120227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21054cbe-a37f-4175-9e61-c8035decf680 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:22.121949 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21054cbe-a37f-4175-9e61-c8035decf680 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:22.132724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1190/2362] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:53:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:53:22.567610 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d29c6-aad4-4243-b080-c594c83a9f78 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:22.569278 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-920d29c6-aad4-4243-b080-c594c83a9f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:22.581871 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920d29c6-aad4-4243-b080-c594c83a9f78 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:22.586696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a6880-2c58-4ef2-9c5e-52ad5f951093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:22.587633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a6880-2c58-4ef2-9c5e-52ad5f951093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:22.587986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a6880-2c58-4ef2-9c5e-52ad5f951093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:22.588264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a6880-2c58-4ef2-9c5e-52ad5f951093 admin admin] PATH_INFO: `/v3/projects/1e771a9d362d4a5983afe499a169c27e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:22.589495 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a6880-2c58-4ef2-9c5e-52ad5f951093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e771a9d362d4a5983afe499a169c27e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:22.591395 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a6880-2c58-4ef2-9c5e-52ad5f951093 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:22.727658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0a0a6880-2c58-4ef2-9c5e-52ad5f951093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e771a9d362d4a5983afe499a169c27e', 'request_id': 'req-0a0a6880-2c58-4ef2-9c5e-52ad5f951093'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:22.733272 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0a0a6880-2c58-4ef2-9c5e-52ad5f951093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e771a9d362d4a5983afe499a169c27e', 'request_id': 'req-0a0a6880-2c58-4ef2-9c5e-52ad5f951093'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:22.799451 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0a0a6880-2c58-4ef2-9c5e-52ad5f951093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e771a9d362d4a5983afe499a169c27e', 'request_id': 'req-0a0a6880-2c58-4ef2-9c5e-52ad5f951093'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:22.811596 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0a0a6880-2c58-4ef2-9c5e-52ad5f951093 admin admin] The token cache is being invalidate because project 1e771a9d362d4a5983afe499a169c27e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:53:22.811890 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0a0a6880-2c58-4ef2-9c5e-52ad5f951093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:22.812547 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0a0a6880-2c58-4ef2-9c5e-52ad5f951093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:22.814125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1173/2363] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:53:22 2022] DELETE /identity/v3/projects/1e771a9d362d4a5983afe499a169c27e => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:53:25.652889 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21054cbe-a37f-4175-9e61-c8035decf680 service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:25.654163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-21054cbe-a37f-4175-9e61-c8035decf680 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:25.682963 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:53:25.746318 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21054cbe-a37f-4175-9e61-c8035decf680 service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:25.804582 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11821af9-785c-4843-84a0-e0f4c55e9099 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2ed13a0c84f128b795002892efc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:25.805453 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11821af9-785c-4843-84a0-e0f4c55e9099 service placement] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:25.805809 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11821af9-785c-4843-84a0-e0f4c55e9099 service placement] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:25.806498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11821af9-785c-4843-84a0-e0f4c55e9099 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:25.808334 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-11821af9-785c-4843-84a0-e0f4c55e9099 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:25.819908 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11821af9-785c-4843-84a0-e0f4c55e9099 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:25.821550 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11821af9-785c-4843-84a0-e0f4c55e9099 service placement] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:25.831374 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1191/2364] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:53:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:53:32.808541 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:53:36.006358 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=57 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:53:44.899226 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a6880-2c58-4ef2-9c5e-52ad5f951093 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:44.900729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-0a0a6880-2c58-4ef2-9c5e-52ad5f951093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:44.965407 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a6880-2c58-4ef2-9c5e-52ad5f951093 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:44.970249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded808e-64f2-401b-a066-d00c1bf896f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:44.971116 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded808e-64f2-401b-a066-d00c1bf896f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:44.971402 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded808e-64f2-401b-a066-d00c1bf896f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:44.971681 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded808e-64f2-401b-a066-d00c1bf896f8 admin admin] PATH_INFO: `/v3/users/6c0be04f176b4529828556fd74507354` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:44.972848 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded808e-64f2-401b-a066-d00c1bf896f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c0be04f176b4529828556fd74507354)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:44.975530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded808e-64f2-401b-a066-d00c1bf896f8 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:45.166595 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1ded808e-64f2-401b-a066-d00c1bf896f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c0be04f176b4529828556fd74507354'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:45.178697 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1ded808e-64f2-401b-a066-d00c1bf896f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c0be04f176b4529828556fd74507354'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:45.232489 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1ded808e-64f2-401b-a066-d00c1bf896f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c0be04f176b4529828556fd74507354'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:45.233130 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1ded808e-64f2-401b-a066-d00c1bf896f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c0be04f176b4529828556fd74507354'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:45.233728 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-1ded808e-64f2-401b-a066-d00c1bf896f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c0be04f176b4529828556fd74507354'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:45.247105 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1174/2365] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:53:44 2022] DELETE /identity/v3/users/6c0be04f176b4529828556fd74507354 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:53:45.251594 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db405581-77b1-41a5-a70b-02f379ca24cc None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:45.252126 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db405581-77b1-41a5-a70b-02f379ca24cc None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:45.252232 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db405581-77b1-41a5-a70b-02f379ca24cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:45.269553 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-db405581-77b1-41a5-a70b-02f379ca24cc None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:53:45.270748 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-db405581-77b1-41a5-a70b-02f379ca24cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:45.336444 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1192/2366] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:53:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:53:45.347609 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded808e-64f2-401b-a066-d00c1bf896f8 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:45.349941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-1ded808e-64f2-401b-a066-d00c1bf896f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:45.415444 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded808e-64f2-401b-a066-d00c1bf896f8 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:45.419688 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ae714-e69a-447b-af0d-b539587f94cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:45.424175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ae714-e69a-447b-af0d-b539587f94cb service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:45.424175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ae714-e69a-447b-af0d-b539587f94cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:45.424175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ae714-e69a-447b-af0d-b539587f94cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:45.430911 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ae714-e69a-447b-af0d-b539587f94cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:45.432334 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ae714-e69a-447b-af0d-b539587f94cb service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:45.441713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1175/2367] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:53:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:53:45.908680 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db405581-77b1-41a5-a70b-02f379ca24cc None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:45.910295 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-db405581-77b1-41a5-a70b-02f379ca24cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:45.927605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db405581-77b1-41a5-a70b-02f379ca24cc None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:45.929235 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77125eef-30cf-4286-acf6-45030d292aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:45.930220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77125eef-30cf-4286-acf6-45030d292aad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:45.930375 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77125eef-30cf-4286-acf6-45030d292aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:45.930653 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77125eef-30cf-4286-acf6-45030d292aad admin admin] PATH_INFO: `/v3/projects/130a0d7717ff460ea894123e5c6cc428` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:45.931831 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77125eef-30cf-4286-acf6-45030d292aad admin admin] RBAC: Authorizing `identity:delete_project(project_id=130a0d7717ff460ea894123e5c6cc428)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:45.933830 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77125eef-30cf-4286-acf6-45030d292aad admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:46.043995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-77125eef-30cf-4286-acf6-45030d292aad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '130a0d7717ff460ea894123e5c6cc428', 'request_id': 'req-77125eef-30cf-4286-acf6-45030d292aad'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:46.049404 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-77125eef-30cf-4286-acf6-45030d292aad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '130a0d7717ff460ea894123e5c6cc428', 'request_id': 'req-77125eef-30cf-4286-acf6-45030d292aad'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:46.055749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-77125eef-30cf-4286-acf6-45030d292aad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '130a0d7717ff460ea894123e5c6cc428', 'request_id': 'req-77125eef-30cf-4286-acf6-45030d292aad'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:46.140634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-77125eef-30cf-4286-acf6-45030d292aad admin admin] The token cache is being invalidate because project 130a0d7717ff460ea894123e5c6cc428 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:53:46.140928 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-77125eef-30cf-4286-acf6-45030d292aad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:46.141583 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-77125eef-30cf-4286-acf6-45030d292aad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:53:46.143126 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1193/2368] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:53:45 2022] DELETE /identity/v3/projects/130a0d7717ff460ea894123e5c6cc428 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:53:55.437931 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:53:55.851802 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ae714-e69a-447b-af0d-b539587f94cb service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:55.867440 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ae714-e69a-447b-af0d-b539587f94cb service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:55.969352 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64adc8d4-c840-46bb-83cf-5d283cfe5625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:55.970094 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64adc8d4-c840-46bb-83cf-5d283cfe5625 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:55.970385 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64adc8d4-c840-46bb-83cf-5d283cfe5625 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:55.970670 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64adc8d4-c840-46bb-83cf-5d283cfe5625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:55.972620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-64adc8d4-c840-46bb-83cf-5d283cfe5625 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:56.046313 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64adc8d4-c840-46bb-83cf-5d283cfe5625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:56.048289 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64adc8d4-c840-46bb-83cf-5d283cfe5625 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:56.061599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1176/2369] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:53:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:53:56.137304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:53:56.244458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a90b4f-b6de-45ed-9b44-1d064947499b None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:56.245066 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a90b4f-b6de-45ed-9b44-1d064947499b None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:56.246710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a90b4f-b6de-45ed-9b44-1d064947499b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:56.267229 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-88a90b4f-b6de-45ed-9b44-1d064947499b None None] MFA Rules not processed for user `c700b0adb9f54947a88c1baa86accb9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:53:56.268450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-88a90b4f-b6de-45ed-9b44-1d064947499b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:56.340766 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1194/2370] 10.222.0.13 () {58 vars in 1009 bytes} [Fri May 27 19:53:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:53:56.355359 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64adc8d4-c840-46bb-83cf-5d283cfe5625 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:56.357067 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-64adc8d4-c840-46bb-83cf-5d283cfe5625 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:56.440209 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64adc8d4-c840-46bb-83cf-5d283cfe5625 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:56.449976 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a95425c-82e4-4a0e-83ce-0439a9cd9b32 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37bc6d0abb454de19e1ab2f46bf4a593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:56.451735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95425c-82e4-4a0e-83ce-0439a9cd9b32 service swift] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:56.452265 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95425c-82e4-4a0e-83ce-0439a9cd9b32 service swift] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:56.453300 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a95425c-82e4-4a0e-83ce-0439a9cd9b32 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:56.463402 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a95425c-82e4-4a0e-83ce-0439a9cd9b32 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:56.467539 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a95425c-82e4-4a0e-83ce-0439a9cd9b32 service swift] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:56.487719 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1177/2371] 10.222.0.13 () {60 vars in 1384 bytes} [Fri May 27 19:53:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 19:53:57.671556 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb2b1f-69a1-4893-8a78-3c57ba9e55c5 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:57.672493 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb2b1f-69a1-4893-8a78-3c57ba9e55c5 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:57.672934 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb2b1f-69a1-4893-8a78-3c57ba9e55c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:57.704498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-03cb2b1f-69a1-4893-8a78-3c57ba9e55c5 None None] MFA Rules not processed for user `c700b0adb9f54947a88c1baa86accb9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:53:57.717286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-03cb2b1f-69a1-4893-8a78-3c57ba9e55c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:57.737317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1195/2372] 10.222.0.13 () {58 vars in 967 bytes} [Fri May 27 19:53:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:53:57.744586 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a95425c-82e4-4a0e-83ce-0439a9cd9b32 service swift] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:57.759370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a95425c-82e4-4a0e-83ce-0439a9cd9b32 service swift] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:57.766286 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f867f6-e506-4628-8dc3-50c4b005035d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37bc6d0abb454de19e1ab2f46bf4a593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:57.767333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f867f6-e506-4628-8dc3-50c4b005035d service swift] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:57.768187 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f867f6-e506-4628-8dc3-50c4b005035d service swift] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:57.768637 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f867f6-e506-4628-8dc3-50c4b005035d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:57.782886 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f867f6-e506-4628-8dc3-50c4b005035d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:57.784794 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f867f6-e506-4628-8dc3-50c4b005035d service swift] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:57.803630 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1178/2373] 10.222.0.13 () {60 vars in 1384 bytes} [Fri May 27 19:53:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 19:53:58.690784 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faeeaff-65da-47f4-b1eb-4274693a64c2 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:58.691325 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faeeaff-65da-47f4-b1eb-4274693a64c2 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:58.691445 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faeeaff-65da-47f4-b1eb-4274693a64c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:58.715701 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-0faeeaff-65da-47f4-b1eb-4274693a64c2 None None] MFA Rules not processed for user `c700b0adb9f54947a88c1baa86accb9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:53:58.717310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-0faeeaff-65da-47f4-b1eb-4274693a64c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:58.725460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1196/2374] 10.222.0.13 () {58 vars in 1009 bytes} [Fri May 27 19:53:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:53:58.737230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f867f6-e506-4628-8dc3-50c4b005035d service swift] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:58.749057 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f867f6-e506-4628-8dc3-50c4b005035d service swift] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:58.754643 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc642b05-6cf7-43fa-a802-99d604d39342 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37bc6d0abb454de19e1ab2f46bf4a593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:58.755304 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc642b05-6cf7-43fa-a802-99d604d39342 service swift] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:58.755594 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc642b05-6cf7-43fa-a802-99d604d39342 service swift] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:58.755909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc642b05-6cf7-43fa-a802-99d604d39342 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:58.765976 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc642b05-6cf7-43fa-a802-99d604d39342 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:58.767583 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc642b05-6cf7-43fa-a802-99d604d39342 service swift] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:58.799894 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1179/2375] 10.222.0.13 () {60 vars in 1384 bytes} [Fri May 27 19:53:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 19:53:59.635088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc437f75-4b65-4d4a-b3cf-b473f04b232d None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:59.635648 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc437f75-4b65-4d4a-b3cf-b473f04b232d None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:59.636290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc437f75-4b65-4d4a-b3cf-b473f04b232d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:59.661810 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-dc437f75-4b65-4d4a-b3cf-b473f04b232d None None] MFA Rules not processed for user `c700b0adb9f54947a88c1baa86accb9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:53:59.663711 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-dc437f75-4b65-4d4a-b3cf-b473f04b232d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:59.677150 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1197/2376] 10.222.0.13 () {58 vars in 967 bytes} [Fri May 27 19:53:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:53:59.691258 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc642b05-6cf7-43fa-a802-99d604d39342 service swift] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:59.703285 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc642b05-6cf7-43fa-a802-99d604d39342 service swift] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:59.710406 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526a72b-2ab3-4584-b370-6a53114ac368 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37bc6d0abb454de19e1ab2f46bf4a593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:59.711470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526a72b-2ab3-4584-b370-6a53114ac368 service swift] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:59.711965 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526a72b-2ab3-4584-b370-6a53114ac368 service swift] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:59.712451 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526a72b-2ab3-4584-b370-6a53114ac368 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:59.743365 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526a72b-2ab3-4584-b370-6a53114ac368 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:59.748360 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526a72b-2ab3-4584-b370-6a53114ac368 service swift] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:59.766639 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:53:59.777958 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1180/2377] 10.222.0.13 () {60 vars in 1384 bytes} [Fri May 27 19:53:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 19:53:59.794585 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9f739-69b6-4165-b9fb-e44f878b6d4b None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:59.794585 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9f739-69b6-4165-b9fb-e44f878b6d4b None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:59.794585 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea9f739-69b6-4165-b9fb-e44f878b6d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:59.830033 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-eea9f739-69b6-4165-b9fb-e44f878b6d4b None None] MFA Rules not processed for user `c700b0adb9f54947a88c1baa86accb9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:53:59.831334 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-eea9f739-69b6-4165-b9fb-e44f878b6d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:53:59.849269 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1198/2378] 10.222.0.13 () {58 vars in 967 bytes} [Fri May 27 19:53:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:53:59.869296 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526a72b-2ab3-4584-b370-6a53114ac368 service swift] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:53:59.880638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526a72b-2ab3-4584-b370-6a53114ac368 service swift] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:53:59.887983 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68bc9c1-372c-4260-b504-6354b29393fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37bc6d0abb454de19e1ab2f46bf4a593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:53:59.889046 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bc9c1-372c-4260-b504-6354b29393fe service swift] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:53:59.892920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bc9c1-372c-4260-b504-6354b29393fe service swift] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:53:59.892920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68bc9c1-372c-4260-b504-6354b29393fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:53:59.900837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68bc9c1-372c-4260-b504-6354b29393fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:53:59.904347 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68bc9c1-372c-4260-b504-6354b29393fe service swift] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:53:59.924685 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1181/2379] 10.222.0.13 () {60 vars in 1384 bytes} [Fri May 27 19:53:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 19:54:03.444510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee69b81-7d1c-4b6b-9704-190063b5a669 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:03.445038 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee69b81-7d1c-4b6b-9704-190063b5a669 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:03.445139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee69b81-7d1c-4b6b-9704-190063b5a669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:03.462385 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-aee69b81-7d1c-4b6b-9704-190063b5a669 None None] MFA Rules not processed for user `a13d2c50c5204c859872e99bc45c6566`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:54:03.463595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-aee69b81-7d1c-4b6b-9704-190063b5a669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:03.555726 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1199/2380] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:54:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:54:05.124424 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e210d-67a6-49fa-be79-c14c508be2c9 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:05.124908 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e210d-67a6-49fa-be79-c14c508be2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:05.125053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e210d-67a6-49fa-be79-c14c508be2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:05.142716 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-257e210d-67a6-49fa-be79-c14c508be2c9 None None] MFA Rules not processed for user `c700b0adb9f54947a88c1baa86accb9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:54:05.143841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-257e210d-67a6-49fa-be79-c14c508be2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:05.151885 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1182/2381] 10.222.0.13 () {58 vars in 1009 bytes} [Fri May 27 19:54:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:54:05.164698 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee69b81-7d1c-4b6b-9704-190063b5a669 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:05.177097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee69b81-7d1c-4b6b-9704-190063b5a669 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:05.182612 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcae43f-9907-4157-bbbe-f37624638262 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37bc6d0abb454de19e1ab2f46bf4a593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:05.184909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcae43f-9907-4157-bbbe-f37624638262 service swift] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:05.184909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcae43f-9907-4157-bbbe-f37624638262 service swift] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:05.184909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcae43f-9907-4157-bbbe-f37624638262 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:05.192720 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcae43f-9907-4157-bbbe-f37624638262 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:05.195133 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcae43f-9907-4157-bbbe-f37624638262 service swift] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:05.204021 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1200/2382] 10.222.0.13 () {60 vars in 1384 bytes} [Fri May 27 19:54:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 19:54:05.243133 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e210d-67a6-49fa-be79-c14c508be2c9 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:05.244577 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-257e210d-67a6-49fa-be79-c14c508be2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:05.311402 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e210d-67a6-49fa-be79-c14c508be2c9 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:05.316086 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a24a6-b773-481c-9f94-f1fcde815e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:05.316982 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a24a6-b773-481c-9f94-f1fcde815e3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:05.317279 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a24a6-b773-481c-9f94-f1fcde815e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:05.317562 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295a24a6-b773-481c-9f94-f1fcde815e3f admin admin] PATH_INFO: `/v3/users/f7870acb5f8b447e8ebb86eaa2ecfcfa` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:05.319306 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295a24a6-b773-481c-9f94-f1fcde815e3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7870acb5f8b447e8ebb86eaa2ecfcfa)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:05.321372 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295a24a6-b773-481c-9f94-f1fcde815e3f admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:05.580341 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-295a24a6-b773-481c-9f94-f1fcde815e3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7870acb5f8b447e8ebb86eaa2ecfcfa'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:05.623992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-295a24a6-b773-481c-9f94-f1fcde815e3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7870acb5f8b447e8ebb86eaa2ecfcfa'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:05.726273 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-295a24a6-b773-481c-9f94-f1fcde815e3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7870acb5f8b447e8ebb86eaa2ecfcfa'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:05.726916 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-295a24a6-b773-481c-9f94-f1fcde815e3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7870acb5f8b447e8ebb86eaa2ecfcfa'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:05.727630 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-295a24a6-b773-481c-9f94-f1fcde815e3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7870acb5f8b447e8ebb86eaa2ecfcfa'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:05.748941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1183/2383] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:54:05 2022] DELETE /identity/v3/users/f7870acb5f8b447e8ebb86eaa2ecfcfa => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:54:05.753311 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb359d3-fedb-4176-a582-4be59b3b200f None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:05.753655 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb359d3-fedb-4176-a582-4be59b3b200f None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:05.753970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb359d3-fedb-4176-a582-4be59b3b200f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:05.775615 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-4fb359d3-fedb-4176-a582-4be59b3b200f None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:54:05.777141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-4fb359d3-fedb-4176-a582-4be59b3b200f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:05.869848 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1201/2384] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:54:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:54:05.879800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a24a6-b773-481c-9f94-f1fcde815e3f admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:05.882064 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-295a24a6-b773-481c-9f94-f1fcde815e3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:05.964588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295a24a6-b773-481c-9f94-f1fcde815e3f admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:05.968909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c056d-5c3f-4db3-89b2-9fb6234c698c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:05.969671 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c056d-5c3f-4db3-89b2-9fb6234c698c service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:05.970031 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c056d-5c3f-4db3-89b2-9fb6234c698c service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:05.970520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c056d-5c3f-4db3-89b2-9fb6234c698c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:05.992240 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c056d-5c3f-4db3-89b2-9fb6234c698c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:05.993849 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6c056d-5c3f-4db3-89b2-9fb6234c698c service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:06.006220 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1184/2385] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:54:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:54:06.337987 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb359d3-fedb-4176-a582-4be59b3b200f None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:06.340524 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-4fb359d3-fedb-4176-a582-4be59b3b200f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:06.356052 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb359d3-fedb-4176-a582-4be59b3b200f None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:06.365482 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74baeafb-4169-47ce-9aa4-7ac8a128ce74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:06.367384 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74baeafb-4169-47ce-9aa4-7ac8a128ce74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:06.367984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74baeafb-4169-47ce-9aa4-7ac8a128ce74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:06.369027 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74baeafb-4169-47ce-9aa4-7ac8a128ce74 admin admin] PATH_INFO: `/v3/projects/96ab6f56bbb34547a1fa2548159e7579` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:06.371029 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74baeafb-4169-47ce-9aa4-7ac8a128ce74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96ab6f56bbb34547a1fa2548159e7579)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:06.373669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74baeafb-4169-47ce-9aa4-7ac8a128ce74 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:06.381976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:54:06.503913 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-74baeafb-4169-47ce-9aa4-7ac8a128ce74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96ab6f56bbb34547a1fa2548159e7579', 'request_id': 'req-74baeafb-4169-47ce-9aa4-7ac8a128ce74'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:06.517548 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-74baeafb-4169-47ce-9aa4-7ac8a128ce74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96ab6f56bbb34547a1fa2548159e7579', 'request_id': 'req-74baeafb-4169-47ce-9aa4-7ac8a128ce74'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:06.530022 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-74baeafb-4169-47ce-9aa4-7ac8a128ce74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96ab6f56bbb34547a1fa2548159e7579', 'request_id': 'req-74baeafb-4169-47ce-9aa4-7ac8a128ce74'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:06.598759 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-74baeafb-4169-47ce-9aa4-7ac8a128ce74 admin admin] The token cache is being invalidate because project 96ab6f56bbb34547a1fa2548159e7579 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:54:06.599362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-74baeafb-4169-47ce-9aa4-7ac8a128ce74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:06.600401 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-74baeafb-4169-47ce-9aa4-7ac8a128ce74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:06.603144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1202/2386] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:54:06 2022] DELETE /identity/v3/projects/96ab6f56bbb34547a1fa2548159e7579 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:54:06.608666 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6cf36-bdd6-483b-a56e-d850233100f0 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:06.609187 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6cf36-bdd6-483b-a56e-d850233100f0 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:06.609276 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6cf36-bdd6-483b-a56e-d850233100f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:06.617906 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:54:06.638819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-9dd6cf36-bdd6-483b-a56e-d850233100f0 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:54:06.641586 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-9dd6cf36-bdd6-483b-a56e-d850233100f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:06.734906 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1185/2387] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:54:06.748335 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34dc6db-9fd0-4ed5-8d15-541d6c131b32 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:06.748335 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34dc6db-9fd0-4ed5-8d15-541d6c131b32 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:06.748712 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34dc6db-9fd0-4ed5-8d15-541d6c131b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:06.790080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-e34dc6db-9fd0-4ed5-8d15-541d6c131b32 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:54:06.792061 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-e34dc6db-9fd0-4ed5-8d15-541d6c131b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:06.808280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1203/2388] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:54:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:54:06.822684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6cf36-bdd6-483b-a56e-d850233100f0 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:06.837008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd6cf36-bdd6-483b-a56e-d850233100f0 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:06.853652 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12855c4-17aa-429e-8915-3d0eb03dea10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:06.854344 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12855c4-17aa-429e-8915-3d0eb03dea10 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:06.854634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12855c4-17aa-429e-8915-3d0eb03dea10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:06.854920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12855c4-17aa-429e-8915-3d0eb03dea10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:06.855664 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12855c4-17aa-429e-8915-3d0eb03dea10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:54:06.855971 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12855c4-17aa-429e-8915-3d0eb03dea10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:06.857702 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12855c4-17aa-429e-8915-3d0eb03dea10 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:06.871759 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1186/2389] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:54:06.876048 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34dc6db-9fd0-4ed5-8d15-541d6c131b32 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:06.889899 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34dc6db-9fd0-4ed5-8d15-541d6c131b32 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:06.894784 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d9e5f-3b5e-4145-af2f-c283afea3208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:06.895452 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d9e5f-3b5e-4145-af2f-c283afea3208 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:06.895740 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d9e5f-3b5e-4145-af2f-c283afea3208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:06.896141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d9e5f-3b5e-4145-af2f-c283afea3208 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:06.898528 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d9e5f-3b5e-4145-af2f-c283afea3208 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:06.900476 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d9e5f-3b5e-4145-af2f-c283afea3208 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:06.984576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12855c4-17aa-429e-8915-3d0eb03dea10 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:06.985090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1204/2390] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:54:06 2022] POST /identity/v3/projects => generated 399 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:54:06.998671 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12855c4-17aa-429e-8915-3d0eb03dea10 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:07.102180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794c7aa-ab23-4368-b2f8-381ea52a1f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:07.103645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794c7aa-ab23-4368-b2f8-381ea52a1f23 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:07.103975 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794c7aa-ab23-4368-b2f8-381ea52a1f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:07.104268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794c7aa-ab23-4368-b2f8-381ea52a1f23 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:07.105049 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794c7aa-ab23-4368-b2f8-381ea52a1f23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:07.107580 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794c7aa-ab23-4368-b2f8-381ea52a1f23 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:07.242734 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d9e5f-3b5e-4145-af2f-c283afea3208 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:07.249602 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1187/2391] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:54:06 2022] POST /identity/v3/users => generated 300 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:54:07.255475 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d9e5f-3b5e-4145-af2f-c283afea3208 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:07.260832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708d86a-6007-4620-955e-92a030f25b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:07.261769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708d86a-6007-4620-955e-92a030f25b1e admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:07.262132 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708d86a-6007-4620-955e-92a030f25b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:07.262497 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708d86a-6007-4620-955e-92a030f25b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:07.263434 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708d86a-6007-4620-955e-92a030f25b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:07.266946 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708d86a-6007-4620-955e-92a030f25b1e admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:07.286420 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1205/2392] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:54:07.290858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794c7aa-ab23-4368-b2f8-381ea52a1f23 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:07.309523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794c7aa-ab23-4368-b2f8-381ea52a1f23 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:07.314584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa28c55-229d-46b4-9407-e4ca79afc199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:07.315402 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa28c55-229d-46b4-9407-e4ca79afc199 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:07.316251 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa28c55-229d-46b4-9407-e4ca79afc199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:07.316619 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa28c55-229d-46b4-9407-e4ca79afc199 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:07.317512 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa28c55-229d-46b4-9407-e4ca79afc199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:07.320416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa28c55-229d-46b4-9407-e4ca79afc199 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:07.337736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708d86a-6007-4620-955e-92a030f25b1e admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:07.347360 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1188/2393] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:54:07 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:54:07.354305 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708d86a-6007-4620-955e-92a030f25b1e admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:07.361657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfdc95-3c72-4fb3-a19e-c31174efaf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:07.363290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfdc95-3c72-4fb3-a19e-c31174efaf9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:07.363793 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfdc95-3c72-4fb3-a19e-c31174efaf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:07.364338 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dfdc95-3c72-4fb3-a19e-c31174efaf9d admin admin] PATH_INFO: `/v3/projects/8350ad63eaa34e24a13dc083f434d871/users/d54196dc5d0d4cdd842ad634635b3b85/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:07.386379 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dfdc95-3c72-4fb3-a19e-c31174efaf9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8350ad63eaa34e24a13dc083f434d871, user_id=d54196dc5d0d4cdd842ad634635b3b85, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:07.392152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dfdc95-3c72-4fb3-a19e-c31174efaf9d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:07.472668 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1206/2394] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:54:07 2022] PUT /identity/v3/projects/8350ad63eaa34e24a13dc083f434d871/users/d54196dc5d0d4cdd842ad634635b3b85/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:54:07.483592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b9e59-8f29-423c-85ce-5df448ea6375 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:07.484200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b9e59-8f29-423c-85ce-5df448ea6375 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:07.484713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b9e59-8f29-423c-85ce-5df448ea6375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:07.517814 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-361b9e59-8f29-423c-85ce-5df448ea6375 None None] MFA Rules not processed for user `d54196dc5d0d4cdd842ad634635b3b85`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:54:07.519210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-361b9e59-8f29-423c-85ce-5df448ea6375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:07.609020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:54:07.614683 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1189/2395] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:54:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:54:07.627582 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfdc95-3c72-4fb3-a19e-c31174efaf9d admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:07.629558 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d5dfdc95-3c72-4fb3-a19e-c31174efaf9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:07.695956 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dfdc95-3c72-4fb3-a19e-c31174efaf9d admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:07.699968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3cff6-e2d9-45ed-adf0-88b944593963 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:07.700755 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3cff6-e2d9-45ed-adf0-88b944593963 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:07.701279 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3cff6-e2d9-45ed-adf0-88b944593963 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:07.701681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f3cff6-e2d9-45ed-adf0-88b944593963 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:07.722885 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3cff6-e2d9-45ed-adf0-88b944593963 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:07.725178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f3cff6-e2d9-45ed-adf0-88b944593963 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:07.745667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1207/2396] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:54:07 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:54:16.523537 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882ac50-f9be-42e2-a6ad-8de93ef379f1 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:16.524096 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882ac50-f9be-42e2-a6ad-8de93ef379f1 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:16.524096 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882ac50-f9be-42e2-a6ad-8de93ef379f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:16.544170 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-8882ac50-f9be-42e2-a6ad-8de93ef379f1 None None] MFA Rules not processed for user `3c9aa6527c13466bbe5dce68a3b81ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:54:16.544170 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-8882ac50-f9be-42e2-a6ad-8de93ef379f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:16.642155 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1190/2397] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:54:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:54:17.683898 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3cff6-e2d9-45ed-adf0-88b944593963 service cinder] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:17.761952 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:54:17.819154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f3cff6-e2d9-45ed-adf0-88b944593963 service cinder] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:17.827616 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050d45a-f9e9-46f5-8471-4dc274a8780c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:17.829343 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050d45a-f9e9-46f5-8471-4dc274a8780c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:17.829863 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050d45a-f9e9-46f5-8471-4dc274a8780c admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:17.830526 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050d45a-f9e9-46f5-8471-4dc274a8780c admin admin] PATH_INFO: `/v3/users/d54196dc5d0d4cdd842ad634635b3b85` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:17.832988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050d45a-f9e9-46f5-8471-4dc274a8780c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d54196dc5d0d4cdd842ad634635b3b85)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:17.837507 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050d45a-f9e9-46f5-8471-4dc274a8780c admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:18.088102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-4050d45a-f9e9-46f5-8471-4dc274a8780c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd54196dc5d0d4cdd842ad634635b3b85'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:18.105555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-4050d45a-f9e9-46f5-8471-4dc274a8780c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd54196dc5d0d4cdd842ad634635b3b85'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:18.179819 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-4050d45a-f9e9-46f5-8471-4dc274a8780c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd54196dc5d0d4cdd842ad634635b3b85'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:18.184603 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-4050d45a-f9e9-46f5-8471-4dc274a8780c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd54196dc5d0d4cdd842ad634635b3b85'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:18.203136 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-4050d45a-f9e9-46f5-8471-4dc274a8780c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd54196dc5d0d4cdd842ad634635b3b85'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:18.219089 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1208/2398] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:54:17 2022] DELETE /identity/v3/users/d54196dc5d0d4cdd842ad634635b3b85 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:54:18.224818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe38e8-b93f-4e92-9cb4-93001327afb9 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:18.225405 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe38e8-b93f-4e92-9cb4-93001327afb9 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:18.225836 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe38e8-b93f-4e92-9cb4-93001327afb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:18.282868 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-59fe38e8-b93f-4e92-9cb4-93001327afb9 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:54:18.285253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-59fe38e8-b93f-4e92-9cb4-93001327afb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:18.373856 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1191/2399] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:54:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:54:18.381398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050d45a-f9e9-46f5-8471-4dc274a8780c admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:18.383091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-4050d45a-f9e9-46f5-8471-4dc274a8780c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:18.464837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050d45a-f9e9-46f5-8471-4dc274a8780c admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:18.470310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc201a3-3de7-4e7c-af9d-359e3f8216d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:18.471022 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc201a3-3de7-4e7c-af9d-359e3f8216d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:18.471387 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc201a3-3de7-4e7c-af9d-359e3f8216d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:18.471738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc201a3-3de7-4e7c-af9d-359e3f8216d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:18.483574 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc201a3-3de7-4e7c-af9d-359e3f8216d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:18.488262 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc201a3-3de7-4e7c-af9d-359e3f8216d5 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:18.496849 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1209/2400] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:54:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:54:19.007345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe38e8-b93f-4e92-9cb4-93001327afb9 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:19.009032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-59fe38e8-b93f-4e92-9cb4-93001327afb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:19.030693 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe38e8-b93f-4e92-9cb4-93001327afb9 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:19.035888 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1f246-ef80-4b9d-ab70-fa48221c1bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:19.037026 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1f246-ef80-4b9d-ab70-fa48221c1bfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:19.037380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1f246-ef80-4b9d-ab70-fa48221c1bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:19.037744 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1f246-ef80-4b9d-ab70-fa48221c1bfc admin admin] PATH_INFO: `/v3/projects/8350ad63eaa34e24a13dc083f434d871` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:19.039250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd1f246-ef80-4b9d-ab70-fa48221c1bfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8350ad63eaa34e24a13dc083f434d871)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:19.041298 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd1f246-ef80-4b9d-ab70-fa48221c1bfc admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:19.164786 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-5cd1f246-ef80-4b9d-ab70-fa48221c1bfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8350ad63eaa34e24a13dc083f434d871', 'request_id': 'req-5cd1f246-ef80-4b9d-ab70-fa48221c1bfc'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:19.173138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-5cd1f246-ef80-4b9d-ab70-fa48221c1bfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8350ad63eaa34e24a13dc083f434d871', 'request_id': 'req-5cd1f246-ef80-4b9d-ab70-fa48221c1bfc'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:19.243371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-5cd1f246-ef80-4b9d-ab70-fa48221c1bfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8350ad63eaa34e24a13dc083f434d871', 'request_id': 'req-5cd1f246-ef80-4b9d-ab70-fa48221c1bfc'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:19.263285 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-5cd1f246-ef80-4b9d-ab70-fa48221c1bfc admin admin] The token cache is being invalidate because project 8350ad63eaa34e24a13dc083f434d871 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:54:19.263613 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-5cd1f246-ef80-4b9d-ab70-fa48221c1bfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:19.264728 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-5cd1f246-ef80-4b9d-ab70-fa48221c1bfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:19.273261 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1192/2401] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:54:19 2022] DELETE /identity/v3/projects/8350ad63eaa34e24a13dc083f434d871 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:54:19.275173 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d4c3a-66e1-4cdd-8cdb-687536432e7e None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:19.275641 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d4c3a-66e1-4cdd-8cdb-687536432e7e None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:19.275723 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d4c3a-66e1-4cdd-8cdb-687536432e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:19.315876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-d58d4c3a-66e1-4cdd-8cdb-687536432e7e None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:54:19.317820 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d58d4c3a-66e1-4cdd-8cdb-687536432e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:19.475966 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:54:19.497380 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1210/2402] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:54:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:54:19.516140 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da26e9-cb4a-441b-bdaf-e3dc933df7e1 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:19.516720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da26e9-cb4a-441b-bdaf-e3dc933df7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:19.517301 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7da26e9-cb4a-441b-bdaf-e3dc933df7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:19.555632 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-a7da26e9-cb4a-441b-bdaf-e3dc933df7e1 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:54:19.559643 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-a7da26e9-cb4a-441b-bdaf-e3dc933df7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:19.571134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1193/2403] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:54:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:54:19.578912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d4c3a-66e1-4cdd-8cdb-687536432e7e None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:19.595283 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d4c3a-66e1-4cdd-8cdb-687536432e7e None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:19.620344 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11dd78-f768-40c2-8e41-747a40f5f276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:19.621624 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11dd78-f768-40c2-8e41-747a40f5f276 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:19.622018 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11dd78-f768-40c2-8e41-747a40f5f276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:19.622517 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11dd78-f768-40c2-8e41-747a40f5f276 admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:19.624217 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11dd78-f768-40c2-8e41-747a40f5f276 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:54:19.624666 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11dd78-f768-40c2-8e41-747a40f5f276 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:19.627019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11dd78-f768-40c2-8e41-747a40f5f276 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:19.653049 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1211/2404] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:54:19.657627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da26e9-cb4a-441b-bdaf-e3dc933df7e1 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:19.673628 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7da26e9-cb4a-441b-bdaf-e3dc933df7e1 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:19.681959 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a504ed-f65d-4d76-88e1-590a963d8a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:19.682759 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a504ed-f65d-4d76-88e1-590a963d8a90 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:19.683123 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a504ed-f65d-4d76-88e1-590a963d8a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:19.683472 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a504ed-f65d-4d76-88e1-590a963d8a90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:19.684252 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a504ed-f65d-4d76-88e1-590a963d8a90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:19.686316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a504ed-f65d-4d76-88e1-590a963d8a90 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:19.835826 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:54:19.874051 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1194/2405] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:54:19 2022] POST /identity/v3/projects => generated 371 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:54:19.875989 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11dd78-f768-40c2-8e41-747a40f5f276 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:19.889932 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11dd78-f768-40c2-8e41-747a40f5f276 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:20.010024 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359326e-2c8c-401b-b93e-da6d9c6b4b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:20.010787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359326e-2c8c-401b-b93e-da6d9c6b4b04 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:20.011078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359326e-2c8c-401b-b93e-da6d9c6b4b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:20.011359 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2359326e-2c8c-401b-b93e-da6d9c6b4b04 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:20.012115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2359326e-2c8c-401b-b93e-da6d9c6b4b04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:20.014654 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2359326e-2c8c-401b-b93e-da6d9c6b4b04 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:20.108830 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1212/2406] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:54:19 2022] POST /identity/v3/users => generated 286 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:54:20.112452 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a504ed-f65d-4d76-88e1-590a963d8a90 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:20.129991 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a504ed-f65d-4d76-88e1-590a963d8a90 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:20.144667 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f52d7-1ae7-4fcb-9499-a79590f49991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:20.145488 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f52d7-1ae7-4fcb-9499-a79590f49991 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:20.145772 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f52d7-1ae7-4fcb-9499-a79590f49991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:20.146052 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f52d7-1ae7-4fcb-9499-a79590f49991 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:20.146794 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f52d7-1ae7-4fcb-9499-a79590f49991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:20.148429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f52d7-1ae7-4fcb-9499-a79590f49991 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:20.172709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1195/2407] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:54:20.173652 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359326e-2c8c-401b-b93e-da6d9c6b4b04 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:20.190071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2359326e-2c8c-401b-b93e-da6d9c6b4b04 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:20.197959 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931909ca-847f-41b9-9520-467b835f500c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:20.198646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931909ca-847f-41b9-9520-467b835f500c admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:20.198941 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931909ca-847f-41b9-9520-467b835f500c admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:20.199216 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931909ca-847f-41b9-9520-467b835f500c admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:20.199955 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931909ca-847f-41b9-9520-467b835f500c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:20.201909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931909ca-847f-41b9-9520-467b835f500c admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:20.214757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1213/2408] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:54:20.219387 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f52d7-1ae7-4fcb-9499-a79590f49991 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:20.230620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f52d7-1ae7-4fcb-9499-a79590f49991 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:20.235561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab169f-58ff-4b31-b87e-b111beaaf785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:20.237090 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab169f-58ff-4b31-b87e-b111beaaf785 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:20.237090 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab169f-58ff-4b31-b87e-b111beaaf785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:20.240764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab169f-58ff-4b31-b87e-b111beaaf785 admin admin] PATH_INFO: `/v3/projects/28f2a07b1f3646209dfdc37dcd2b755e/users/67ef94416f1741a88c1e18106cad8b2c/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:20.254701 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab169f-58ff-4b31-b87e-b111beaaf785 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28f2a07b1f3646209dfdc37dcd2b755e, user_id=67ef94416f1741a88c1e18106cad8b2c, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:20.257800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab169f-58ff-4b31-b87e-b111beaaf785 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:20.348375 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1196/2409] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:54:20 2022] PUT /identity/v3/projects/28f2a07b1f3646209dfdc37dcd2b755e/users/67ef94416f1741a88c1e18106cad8b2c/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:54:20.353413 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4f1ec-b628-4005-839c-e4420c99a99b None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:20.353918 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4f1ec-b628-4005-839c-e4420c99a99b None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:20.354120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4f1ec-b628-4005-839c-e4420c99a99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:20.394920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-fdd4f1ec-b628-4005-839c-e4420c99a99b None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:54:20.398056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-fdd4f1ec-b628-4005-839c-e4420c99a99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:20.540355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1214/2410] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:54:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:54:20.553500 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab169f-58ff-4b31-b87e-b111beaaf785 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:20.557922 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-10ab169f-58ff-4b31-b87e-b111beaaf785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:20.650610 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab169f-58ff-4b31-b87e-b111beaaf785 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:20.654561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916ecbf-de17-4f12-a301-056eac780b93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:20.655455 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916ecbf-de17-4f12-a301-056eac780b93 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:20.655757 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916ecbf-de17-4f12-a301-056eac780b93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:20.656102 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916ecbf-de17-4f12-a301-056eac780b93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:20.668306 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1916ecbf-de17-4f12-a301-056eac780b93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:20.669895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1916ecbf-de17-4f12-a301-056eac780b93 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:20.682075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1197/2411] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:54:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:54:27.315587 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28c0fd-31cb-4074-babe-2861b7c9a618 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:27.316137 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28c0fd-31cb-4074-babe-2861b7c9a618 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:27.316137 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a28c0fd-31cb-4074-babe-2861b7c9a618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:27.348511 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-2a28c0fd-31cb-4074-babe-2861b7c9a618 None None] MFA Rules not processed for user `c700b0adb9f54947a88c1baa86accb9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:54:27.350692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-2a28c0fd-31cb-4074-babe-2861b7c9a618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:27.442749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1215/2412] 10.222.0.13 () {58 vars in 967 bytes} [Fri May 27 19:54:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:54:27.462875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916ecbf-de17-4f12-a301-056eac780b93 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:27.462875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-1916ecbf-de17-4f12-a301-056eac780b93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:27.543826 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916ecbf-de17-4f12-a301-056eac780b93 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:27.549629 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285ecd8-f703-4347-aca8-46d9f760edf0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37bc6d0abb454de19e1ab2f46bf4a593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:27.550303 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285ecd8-f703-4347-aca8-46d9f760edf0 service swift] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:27.550592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285ecd8-f703-4347-aca8-46d9f760edf0 service swift] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:27.550879 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285ecd8-f703-4347-aca8-46d9f760edf0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:27.562242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285ecd8-f703-4347-aca8-46d9f760edf0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:27.564269 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285ecd8-f703-4347-aca8-46d9f760edf0 service swift] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:27.579570 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1198/2413] 10.222.0.13 () {60 vars in 1384 bytes} [Fri May 27 19:54:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 19:54:29.317495 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f86b2f-ae9a-44ad-9e13-3f081244a201 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:29.318995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f86b2f-ae9a-44ad-9e13-3f081244a201 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:29.319450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f86b2f-ae9a-44ad-9e13-3f081244a201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:29.341994 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:54:29.367082 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-c0f86b2f-ae9a-44ad-9e13-3f081244a201 None None] MFA Rules not processed for user `67ef94416f1741a88c1e18106cad8b2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:54:29.368565 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-c0f86b2f-ae9a-44ad-9e13-3f081244a201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:29.437184 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1216/2414] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:54:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:54:29.442522 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285ecd8-f703-4347-aca8-46d9f760edf0 service swift] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:29.454731 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285ecd8-f703-4347-aca8-46d9f760edf0 service swift] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:29.459612 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d502c-e52c-4f1b-a183-04f526e19635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:29.460262 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d502c-e52c-4f1b-a183-04f526e19635 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:29.460548 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d502c-e52c-4f1b-a183-04f526e19635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:29.460816 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5d502c-e52c-4f1b-a183-04f526e19635 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:29.461533 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d502c-e52c-4f1b-a183-04f526e19635 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:29.463507 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5d502c-e52c-4f1b-a183-04f526e19635 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:29.535102 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1199/2415] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:54:29 2022] POST /identity/v3/projects => generated 371 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:54:29.539532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f86b2f-ae9a-44ad-9e13-3f081244a201 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:29.550479 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f86b2f-ae9a-44ad-9e13-3f081244a201 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:29.612885 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69b2e4-f986-47b3-a550-7b62df3f821f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:29.614604 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69b2e4-f986-47b3-a550-7b62df3f821f admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:29.615027 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69b2e4-f986-47b3-a550-7b62df3f821f admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:29.619731 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c69b2e4-f986-47b3-a550-7b62df3f821f admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:29.619731 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69b2e4-f986-47b3-a550-7b62df3f821f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:29.619731 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c69b2e4-f986-47b3-a550-7b62df3f821f admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:29.693976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1217/2416] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:54:29 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:54:29.697839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d502c-e52c-4f1b-a183-04f526e19635 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:29.708291 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5d502c-e52c-4f1b-a183-04f526e19635 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:29.719107 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d86b6-d1a8-4a9a-98d6-b89f1760c2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:29.719797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d86b6-d1a8-4a9a-98d6-b89f1760c2f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:29.720101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d86b6-d1a8-4a9a-98d6-b89f1760c2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:29.720384 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d86b6-d1a8-4a9a-98d6-b89f1760c2f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:29.721163 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d86b6-d1a8-4a9a-98d6-b89f1760c2f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:29.723768 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d86b6-d1a8-4a9a-98d6-b89f1760c2f2 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:29.735455 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1200/2417] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:54:29.740470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69b2e4-f986-47b3-a550-7b62df3f821f admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:29.751458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c69b2e4-f986-47b3-a550-7b62df3f821f admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:29.756306 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe93502-e930-4321-bd79-9b1cb9653dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:29.757238 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe93502-e930-4321-bd79-9b1cb9653dcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:29.757752 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe93502-e930-4321-bd79-9b1cb9653dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:29.757879 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe93502-e930-4321-bd79-9b1cb9653dcf admin admin] PATH_INFO: `/v3/projects/aefcb4072a0241bca4c1416c113f480a/users/6cb915e3b95d49a08644ac252c9edb6e/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:29.773824 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe93502-e930-4321-bd79-9b1cb9653dcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=aefcb4072a0241bca4c1416c113f480a, user_id=6cb915e3b95d49a08644ac252c9edb6e, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:29.776519 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe93502-e930-4321-bd79-9b1cb9653dcf admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:29.838720 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1218/2418] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:54:29 2022] PUT /identity/v3/projects/aefcb4072a0241bca4c1416c113f480a/users/6cb915e3b95d49a08644ac252c9edb6e/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:54:29.841846 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d86b6-d1a8-4a9a-98d6-b89f1760c2f2 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:29.855402 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d86b6-d1a8-4a9a-98d6-b89f1760c2f2 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:29.873771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:54:29.914100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de76b8c-42a5-4a56-9e5e-45009dfd431d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:29.914899 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de76b8c-42a5-4a56-9e5e-45009dfd431d admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:29.915247 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de76b8c-42a5-4a56-9e5e-45009dfd431d admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:29.915719 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de76b8c-42a5-4a56-9e5e-45009dfd431d admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:29.916624 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de76b8c-42a5-4a56-9e5e-45009dfd431d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:29.918867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de76b8c-42a5-4a56-9e5e-45009dfd431d admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:29.933021 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1201/2419] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:54:29 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:54:29.934578 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe93502-e930-4321-bd79-9b1cb9653dcf admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:29.946147 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe93502-e930-4321-bd79-9b1cb9653dcf admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:29.950925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ad3d5-f052-4c81-b6e7-1e5711b270c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:29.951958 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ad3d5-f052-4c81-b6e7-1e5711b270c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:29.952144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ad3d5-f052-4c81-b6e7-1e5711b270c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:29.952431 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ad3d5-f052-4c81-b6e7-1e5711b270c3 admin admin] PATH_INFO: `/v3/projects/aefcb4072a0241bca4c1416c113f480a/users/6cb915e3b95d49a08644ac252c9edb6e/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:29.954488 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4ad3d5-f052-4c81-b6e7-1e5711b270c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aefcb4072a0241bca4c1416c113f480a, user_id=6cb915e3b95d49a08644ac252c9edb6e, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:29.957952 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4ad3d5-f052-4c81-b6e7-1e5711b270c3 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:29.968690 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1219/2420] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:54:29 2022] PUT /identity/v3/projects/aefcb4072a0241bca4c1416c113f480a/users/6cb915e3b95d49a08644ac252c9edb6e/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:54:31.256830 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de76b8c-42a5-4a56-9e5e-45009dfd431d admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:31.259675 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-8de76b8c-42a5-4a56-9e5e-45009dfd431d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:31.279294 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de76b8c-42a5-4a56-9e5e-45009dfd431d admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:31.341704 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47b773-8141-4379-b7e6-d717e48972fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:31.342517 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47b773-8141-4379-b7e6-d717e48972fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:31.342898 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47b773-8141-4379-b7e6-d717e48972fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:31.343270 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47b773-8141-4379-b7e6-d717e48972fb admin admin] PATH_INFO: `/v3/users/83d3494760b84bc1aad951bba11caea3` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:31.345116 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47b773-8141-4379-b7e6-d717e48972fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=83d3494760b84bc1aad951bba11caea3)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:31.350107 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47b773-8141-4379-b7e6-d717e48972fb admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:31.573596 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-2d47b773-8141-4379-b7e6-d717e48972fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83d3494760b84bc1aad951bba11caea3'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:31.608918 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-2d47b773-8141-4379-b7e6-d717e48972fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83d3494760b84bc1aad951bba11caea3'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:31.653541 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-2d47b773-8141-4379-b7e6-d717e48972fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83d3494760b84bc1aad951bba11caea3'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:31.654247 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-2d47b773-8141-4379-b7e6-d717e48972fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83d3494760b84bc1aad951bba11caea3'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:31.654966 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-2d47b773-8141-4379-b7e6-d717e48972fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83d3494760b84bc1aad951bba11caea3'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:31.682662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1202/2421] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:54:31 2022] DELETE /identity/v3/users/83d3494760b84bc1aad951bba11caea3 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:54:32.198215 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ad3d5-f052-4c81-b6e7-1e5711b270c3 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:32.198215 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-0a4ad3d5-f052-4c81-b6e7-1e5711b270c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:32.216762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ad3d5-f052-4c81-b6e7-1e5711b270c3 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:32.327336 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:54:32.342812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806c98f8-4027-4427-88e2-00e901594bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:32.344287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c98f8-4027-4427-88e2-00e901594bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:32.344776 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c98f8-4027-4427-88e2-00e901594bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:32.345325 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806c98f8-4027-4427-88e2-00e901594bb6 admin admin] PATH_INFO: `/v3/projects/729158c0db83438a9b762aae3f0009a2` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:32.351885 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806c98f8-4027-4427-88e2-00e901594bb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=729158c0db83438a9b762aae3f0009a2)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:32.351885 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806c98f8-4027-4427-88e2-00e901594bb6 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:32.471783 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-806c98f8-4027-4427-88e2-00e901594bb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '729158c0db83438a9b762aae3f0009a2', 'request_id': 'req-806c98f8-4027-4427-88e2-00e901594bb6'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:32.481869 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-806c98f8-4027-4427-88e2-00e901594bb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '729158c0db83438a9b762aae3f0009a2', 'request_id': 'req-806c98f8-4027-4427-88e2-00e901594bb6'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:32.490051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-806c98f8-4027-4427-88e2-00e901594bb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '729158c0db83438a9b762aae3f0009a2', 'request_id': 'req-806c98f8-4027-4427-88e2-00e901594bb6'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:32.562971 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-806c98f8-4027-4427-88e2-00e901594bb6 admin admin] The token cache is being invalidate because project 729158c0db83438a9b762aae3f0009a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:54:32.563148 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-806c98f8-4027-4427-88e2-00e901594bb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:32.564068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-806c98f8-4027-4427-88e2-00e901594bb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:54:32.567755 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1220/2422] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:54:32 2022] DELETE /identity/v3/projects/729158c0db83438a9b762aae3f0009a2 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:54:36.688817 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96953a12-e1d0-4135-b8a2-62b607ab473c None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:36.689392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96953a12-e1d0-4135-b8a2-62b607ab473c None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:36.689523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96953a12-e1d0-4135-b8a2-62b607ab473c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:36.718716 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-96953a12-e1d0-4135-b8a2-62b607ab473c None None] MFA Rules not processed for user `6cb915e3b95d49a08644ac252c9edb6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:54:36.720156 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-96953a12-e1d0-4135-b8a2-62b607ab473c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:36.751345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:54:36.797601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1203/2423] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:54:36 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:54:36.812559 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806c98f8-4027-4427-88e2-00e901594bb6 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:54:36.814067 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-806c98f8-4027-4427-88e2-00e901594bb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:36.893251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806c98f8-4027-4427-88e2-00e901594bb6 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:54:36.899516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace635d6-4a0d-41aa-9c93-31e691892d7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:54:36.900320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace635d6-4a0d-41aa-9c93-31e691892d7f service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:36.900480 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace635d6-4a0d-41aa-9c93-31e691892d7f service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:36.900762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace635d6-4a0d-41aa-9c93-31e691892d7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:36.902734 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ace635d6-4a0d-41aa-9c93-31e691892d7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:36.963481 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace635d6-4a0d-41aa-9c93-31e691892d7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:54:36.965151 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace635d6-4a0d-41aa-9c93-31e691892d7f service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:54:36.974967 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1221/2424] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:54:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:54:39.122748 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ec4dc-6c49-49cf-8ff8-47b463d0c4d1 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:39.123516 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ec4dc-6c49-49cf-8ff8-47b463d0c4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:39.123516 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ec4dc-6c49-49cf-8ff8-47b463d0c4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:39.142661 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-006ec4dc-6c49-49cf-8ff8-47b463d0c4d1 None None] MFA Rules not processed for user `67ef94416f1741a88c1e18106cad8b2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:54:39.144541 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-006ec4dc-6c49-49cf-8ff8-47b463d0c4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:39.151866 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1204/2425] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:54:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:54:46.970215 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:54:49.136567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=20 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:54:59.202841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595f605-7271-4436-a577-bc1102c26759 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:54:59.203532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595f605-7271-4436-a577-bc1102c26759 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:54:59.203532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595f605-7271-4436-a577-bc1102c26759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:54:59.226126 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-b595f605-7271-4436-a577-bc1102c26759 None None] MFA Rules not processed for user `ed8754ddcfd54c24ac923db0fedc1606`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:54:59.227398 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-b595f605-7271-4436-a577-bc1102c26759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:54:59.295250 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1222/2426] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:54:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:55:09.288173 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=25 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:55:15.735709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ec4dc-6c49-49cf-8ff8-47b463d0c4d1 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:55:15.737308 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-006ec4dc-6c49-49cf-8ff8-47b463d0c4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:55:15.794140 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ec4dc-6c49-49cf-8ff8-47b463d0c4d1 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:55:15.797889 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6b2e2-3158-4dff-b03a-3ce2114b6167 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:55:15.798538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6b2e2-3158-4dff-b03a-3ce2114b6167 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:55:15.798826 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6b2e2-3158-4dff-b03a-3ce2114b6167 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:55:15.799106 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6b2e2-3158-4dff-b03a-3ce2114b6167 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:55:15.800879 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-aca6b2e2-3158-4dff-b03a-3ce2114b6167 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:55:15.815446 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6b2e2-3158-4dff-b03a-3ce2114b6167 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:55:15.816880 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6b2e2-3158-4dff-b03a-3ce2114b6167 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:55:15.827637 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1205/2427] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:55:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:55:16.951278 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595f605-7271-4436-a577-bc1102c26759 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:55:16.952679 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-b595f605-7271-4436-a577-bc1102c26759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:55:16.964654 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595f605-7271-4436-a577-bc1102c26759 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:55:16.968743 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c379a77-9b1a-424b-bc5a-8af01fc0070f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:55:16.969509 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c379a77-9b1a-424b-bc5a-8af01fc0070f service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:55:16.969942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c379a77-9b1a-424b-bc5a-8af01fc0070f service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:55:16.970218 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c379a77-9b1a-424b-bc5a-8af01fc0070f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:55:16.971934 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-5c379a77-9b1a-424b-bc5a-8af01fc0070f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:55:16.985716 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c379a77-9b1a-424b-bc5a-8af01fc0070f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:55:16.987137 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c379a77-9b1a-424b-bc5a-8af01fc0070f service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:55:16.997669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1223/2428] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:55:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:55:17.289184 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6b2e2-3158-4dff-b03a-3ce2114b6167 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:55:17.297669 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6b2e2-3158-4dff-b03a-3ce2114b6167 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:55:17.301865 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21321c6d-c79d-42e9-bfed-00bed2fca2fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:55:17.302496 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21321c6d-c79d-42e9-bfed-00bed2fca2fb service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:55:17.302778 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21321c6d-c79d-42e9-bfed-00bed2fca2fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:55:17.303053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21321c6d-c79d-42e9-bfed-00bed2fca2fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:55:17.311957 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21321c6d-c79d-42e9-bfed-00bed2fca2fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:55:17.313451 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21321c6d-c79d-42e9-bfed-00bed2fca2fb service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:55:17.323434 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1206/2429] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:55:17 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:55:25.531477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c379a77-9b1a-424b-bc5a-8af01fc0070f service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:55:25.532856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-5c379a77-9b1a-424b-bc5a-8af01fc0070f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:55:25.590883 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c379a77-9b1a-424b-bc5a-8af01fc0070f service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:55:25.594485 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5620043c-7702-4c5e-af90-c27fb6d88e7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:55:25.595130 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620043c-7702-4c5e-af90-c27fb6d88e7f service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:55:25.595414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620043c-7702-4c5e-af90-c27fb6d88e7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:55:25.595689 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620043c-7702-4c5e-af90-c27fb6d88e7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:55:25.597883 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-5620043c-7702-4c5e-af90-c27fb6d88e7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:55:25.657259 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620043c-7702-4c5e-af90-c27fb6d88e7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:55:25.658728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5620043c-7702-4c5e-af90-c27fb6d88e7f service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:55:25.667945 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1224/2430] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:55:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:55:27.319015 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=32 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:55:33.225329 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21321c6d-c79d-42e9-bfed-00bed2fca2fb service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:55:33.248498 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21321c6d-c79d-42e9-bfed-00bed2fca2fb service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:55:33.260516 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9461e84-b77a-4915-bc4f-4961cbabd85a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:55:33.260516 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9461e84-b77a-4915-bc4f-4961cbabd85a service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:55:33.263823 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9461e84-b77a-4915-bc4f-4961cbabd85a service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:55:33.264497 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9461e84-b77a-4915-bc4f-4961cbabd85a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:55:33.276450 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-a9461e84-b77a-4915-bc4f-4961cbabd85a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:55:33.306870 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9461e84-b77a-4915-bc4f-4961cbabd85a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:55:33.308692 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9461e84-b77a-4915-bc4f-4961cbabd85a service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:55:33.334695 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1207/2431] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:55:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:55:35.663630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=54 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:55:37.725215 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc10eca-a7c5-46eb-9987-f07ec7a97594 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:55:37.728624 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc10eca-a7c5-46eb-9987-f07ec7a97594 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:55:37.728624 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc10eca-a7c5-46eb-9987-f07ec7a97594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:55:37.743794 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-fcc10eca-a7c5-46eb-9987-f07ec7a97594 None None] MFA Rules not processed for user `3c9aa6527c13466bbe5dce68a3b81ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:55:37.745025 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-fcc10eca-a7c5-46eb-9987-f07ec7a97594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:55:37.752135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1225/2432] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:55:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:55:38.056380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9461e84-b77a-4915-bc4f-4961cbabd85a service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:55:38.065100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9461e84-b77a-4915-bc4f-4961cbabd85a service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:55:38.068895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf090d7-0bf8-414c-b2c4-9d652ec4adbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:55:38.069583 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf090d7-0bf8-414c-b2c4-9d652ec4adbb service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:55:38.069872 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf090d7-0bf8-414c-b2c4-9d652ec4adbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:55:38.070154 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf090d7-0bf8-414c-b2c4-9d652ec4adbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:55:38.071947 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-0bf090d7-0bf8-414c-b2c4-9d652ec4adbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:55:38.081680 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf090d7-0bf8-414c-b2c4-9d652ec4adbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:55:38.083579 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf090d7-0bf8-414c-b2c4-9d652ec4adbb service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:55:38.094142 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1208/2433] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:55:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:55:38.528749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc10eca-a7c5-46eb-9987-f07ec7a97594 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:55:38.530263 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-fcc10eca-a7c5-46eb-9987-f07ec7a97594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:55:38.600931 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc10eca-a7c5-46eb-9987-f07ec7a97594 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:55:38.605914 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98736e36-139b-41b2-988b-5ea4cde938d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2ed13a0c84f128b795002892efc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:55:38.607841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98736e36-139b-41b2-988b-5ea4cde938d6 service placement] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:55:38.607841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98736e36-139b-41b2-988b-5ea4cde938d6 service placement] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:55:38.607841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98736e36-139b-41b2-988b-5ea4cde938d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:55:38.608920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-98736e36-139b-41b2-988b-5ea4cde938d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:55:38.621061 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98736e36-139b-41b2-988b-5ea4cde938d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:55:38.622699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98736e36-139b-41b2-988b-5ea4cde938d6 service placement] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:55:38.633314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1226/2434] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:55:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:55:42.802744 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d27f4-dec8-4984-ab4f-cfce6b1ea0ae None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:55:42.803620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d27f4-dec8-4984-ab4f-cfce6b1ea0ae None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:55:42.804094 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642d27f4-dec8-4984-ab4f-cfce6b1ea0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:55:42.822751 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-642d27f4-dec8-4984-ab4f-cfce6b1ea0ae None None] MFA Rules not processed for user `a13d2c50c5204c859872e99bc45c6566`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:55:42.823961 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-642d27f4-dec8-4984-ab4f-cfce6b1ea0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:55:42.881304 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1209/2435] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:55:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:55:43.420638 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe7b3a-62f7-4ba5-ac71-4fd7040645e0 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:55:43.421171 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe7b3a-62f7-4ba5-ac71-4fd7040645e0 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:55:43.422274 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe7b3a-62f7-4ba5-ac71-4fd7040645e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:55:43.449531 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-9bfe7b3a-62f7-4ba5-ac71-4fd7040645e0 None None] MFA Rules not processed for user `c700b0adb9f54947a88c1baa86accb9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:55:43.451122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-9bfe7b3a-62f7-4ba5-ac71-4fd7040645e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:55:43.518645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1227/2436] 10.222.0.13 () {58 vars in 967 bytes} [Fri May 27 19:55:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:55:43.665798 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d27f4-dec8-4984-ab4f-cfce6b1ea0ae None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:55:43.667307 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-642d27f4-dec8-4984-ab4f-cfce6b1ea0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:55:43.758990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642d27f4-dec8-4984-ab4f-cfce6b1ea0ae None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:55:43.770480 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00ee40-d942-46ed-b7c2-f0514ae4fbe5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37bc6d0abb454de19e1ab2f46bf4a593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:55:43.771168 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00ee40-d942-46ed-b7c2-f0514ae4fbe5 service swift] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:55:43.771457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00ee40-d942-46ed-b7c2-f0514ae4fbe5 service swift] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:55:43.771991 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b00ee40-d942-46ed-b7c2-f0514ae4fbe5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:55:43.783548 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b00ee40-d942-46ed-b7c2-f0514ae4fbe5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:55:43.786137 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b00ee40-d942-46ed-b7c2-f0514ae4fbe5 service swift] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:55:43.796520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1210/2437] 10.222.0.13 () {60 vars in 1384 bytes} [Fri May 27 19:55:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 19:55:48.909532 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe7b3a-62f7-4ba5-ac71-4fd7040645e0 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:55:48.914630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-9bfe7b3a-62f7-4ba5-ac71-4fd7040645e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:55:48.965586 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe7b3a-62f7-4ba5-ac71-4fd7040645e0 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:55:48.976426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c1f12-1435-4b12-958c-a665b8558f68 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2ed13a0c84f128b795002892efc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:55:48.977351 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c1f12-1435-4b12-958c-a665b8558f68 service placement] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:55:48.977698 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c1f12-1435-4b12-958c-a665b8558f68 service placement] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:55:48.983103 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38c1f12-1435-4b12-958c-a665b8558f68 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:55:48.983103 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-b38c1f12-1435-4b12-958c-a665b8558f68 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:55:48.995698 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c1f12-1435-4b12-958c-a665b8558f68 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:55:48.997674 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38c1f12-1435-4b12-958c-a665b8558f68 service placement] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:55:49.010229 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1228/2438] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:55:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:55:53.647462 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00ee40-d942-46ed-b7c2-f0514ae4fbe5 service swift] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:55:53.648990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-4b00ee40-d942-46ed-b7c2-f0514ae4fbe5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:55:53.718550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b00ee40-d942-46ed-b7c2-f0514ae4fbe5 service swift] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:55:53.723520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2cbd71-375d-4307-ab27-7f8e04f1733b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:55:53.724405 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cbd71-375d-4307-ab27-7f8e04f1733b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:55:53.724696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cbd71-375d-4307-ab27-7f8e04f1733b admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:55:53.725006 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2cbd71-375d-4307-ab27-7f8e04f1733b admin admin] PATH_INFO: `/v3/users/c0f12003099f4f80bb89c2a335d231ca` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:55:53.726214 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2cbd71-375d-4307-ab27-7f8e04f1733b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0f12003099f4f80bb89c2a335d231ca)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:55:53.728159 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2cbd71-375d-4307-ab27-7f8e04f1733b admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:55:53.740166 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:55:53.908323 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-8c2cbd71-375d-4307-ab27-7f8e04f1733b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0f12003099f4f80bb89c2a335d231ca'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:55:53.919884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-8c2cbd71-375d-4307-ab27-7f8e04f1733b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0f12003099f4f80bb89c2a335d231ca'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:55:53.985671 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-8c2cbd71-375d-4307-ab27-7f8e04f1733b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0f12003099f4f80bb89c2a335d231ca'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:55:53.986304 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-8c2cbd71-375d-4307-ab27-7f8e04f1733b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0f12003099f4f80bb89c2a335d231ca'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:55:53.986918 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-8c2cbd71-375d-4307-ab27-7f8e04f1733b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0f12003099f4f80bb89c2a335d231ca'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:55:54.000987 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1211/2439] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:55:53 2022] DELETE /identity/v3/users/c0f12003099f4f80bb89c2a335d231ca => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:55:54.323311 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c1f12-1435-4b12-958c-a665b8558f68 service placement] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:55:54.336329 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38c1f12-1435-4b12-958c-a665b8558f68 service placement] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:55:54.411014 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597ffc6-ad3c-4eb8-975e-923a98cb1e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:55:54.412060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597ffc6-ad3c-4eb8-975e-923a98cb1e5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:55:54.412351 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597ffc6-ad3c-4eb8-975e-923a98cb1e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:55:54.412630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597ffc6-ad3c-4eb8-975e-923a98cb1e5d admin admin] PATH_INFO: `/v3/projects/0ca4fe16a0204d0db4289e293771581b` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:55:54.414294 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597ffc6-ad3c-4eb8-975e-923a98cb1e5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ca4fe16a0204d0db4289e293771581b)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:55:54.416384 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597ffc6-ad3c-4eb8-975e-923a98cb1e5d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:55:54.436790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:55:54.515876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d597ffc6-ad3c-4eb8-975e-923a98cb1e5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ca4fe16a0204d0db4289e293771581b', 'request_id': 'req-d597ffc6-ad3c-4eb8-975e-923a98cb1e5d'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:55:54.527860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d597ffc6-ad3c-4eb8-975e-923a98cb1e5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ca4fe16a0204d0db4289e293771581b', 'request_id': 'req-d597ffc6-ad3c-4eb8-975e-923a98cb1e5d'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:55:54.535552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d597ffc6-ad3c-4eb8-975e-923a98cb1e5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ca4fe16a0204d0db4289e293771581b', 'request_id': 'req-d597ffc6-ad3c-4eb8-975e-923a98cb1e5d'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:55:54.620519 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d597ffc6-ad3c-4eb8-975e-923a98cb1e5d admin admin] The token cache is being invalidate because project 0ca4fe16a0204d0db4289e293771581b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:55:54.620862 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d597ffc6-ad3c-4eb8-975e-923a98cb1e5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:55:54.621559 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d597ffc6-ad3c-4eb8-975e-923a98cb1e5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:55:54.623629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1229/2440] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:55:54 2022] DELETE /identity/v3/projects/0ca4fe16a0204d0db4289e293771581b => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:55:54.629393 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2cbd71-375d-4307-ab27-7f8e04f1733b admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:55:54.630965 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-8c2cbd71-375d-4307-ab27-7f8e04f1733b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:55:54.655959 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2cbd71-375d-4307-ab27-7f8e04f1733b admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:55:54.686325 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597ffc6-ad3c-4eb8-975e-923a98cb1e5d admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:55:54.687701 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d597ffc6-ad3c-4eb8-975e-923a98cb1e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:55:54.740814 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69b61b-f99d-4876-83c7-19e0ef0da315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:55:54.741906 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69b61b-f99d-4876-83c7-19e0ef0da315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:55:54.742334 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69b61b-f99d-4876-83c7-19e0ef0da315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:55:54.742625 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69b61b-f99d-4876-83c7-19e0ef0da315 admin admin] PATH_INFO: `/v3/users/a4327447184a413380ad998dadcab5d9` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:55:54.743979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69b61b-f99d-4876-83c7-19e0ef0da315 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4327447184a413380ad998dadcab5d9)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:55:54.746082 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69b61b-f99d-4876-83c7-19e0ef0da315 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:55:54.774594 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597ffc6-ad3c-4eb8-975e-923a98cb1e5d admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:55:54.781534 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3909a2-dfa1-4097-8003-bd5698fc7deb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:55:54.782474 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3909a2-dfa1-4097-8003-bd5698fc7deb service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:55:54.782758 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3909a2-dfa1-4097-8003-bd5698fc7deb service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:55:54.783043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3909a2-dfa1-4097-8003-bd5698fc7deb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:55:54.785678 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-4d3909a2-dfa1-4097-8003-bd5698fc7deb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:55:54.799613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3909a2-dfa1-4097-8003-bd5698fc7deb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:55:54.802695 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3909a2-dfa1-4097-8003-bd5698fc7deb service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:55:54.814141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1230/2441] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:55:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:55:54.957400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0e69b61b-f99d-4876-83c7-19e0ef0da315 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4327447184a413380ad998dadcab5d9'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:55:54.973486 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0e69b61b-f99d-4876-83c7-19e0ef0da315 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4327447184a413380ad998dadcab5d9'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:55:55.128440 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0e69b61b-f99d-4876-83c7-19e0ef0da315 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4327447184a413380ad998dadcab5d9'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:55:55.129593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0e69b61b-f99d-4876-83c7-19e0ef0da315 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4327447184a413380ad998dadcab5d9'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:55:55.130205 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0e69b61b-f99d-4876-83c7-19e0ef0da315 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4327447184a413380ad998dadcab5d9'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:55:55.143759 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1212/2442] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:55:54 2022] DELETE /identity/v3/users/a4327447184a413380ad998dadcab5d9 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:55:55.496924 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3909a2-dfa1-4097-8003-bd5698fc7deb service nova] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:55:55.498736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-4d3909a2-dfa1-4097-8003-bd5698fc7deb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:55:55.517907 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3909a2-dfa1-4097-8003-bd5698fc7deb service nova] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:55:55.588920 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e496d-87aa-4a70-b6f1-7317ef705845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:55:55.589867 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e496d-87aa-4a70-b6f1-7317ef705845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:55:55.590138 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e496d-87aa-4a70-b6f1-7317ef705845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:55:55.590413 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e496d-87aa-4a70-b6f1-7317ef705845 admin admin] PATH_INFO: `/v3/projects/8019c7f3ebd94943a22d342c2ba83aa6` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:55:55.591638 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e496d-87aa-4a70-b6f1-7317ef705845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8019c7f3ebd94943a22d342c2ba83aa6)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:55:55.593648 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e496d-87aa-4a70-b6f1-7317ef705845 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:55:55.713456 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:55:55.727114 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-327e496d-87aa-4a70-b6f1-7317ef705845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8019c7f3ebd94943a22d342c2ba83aa6', 'request_id': 'req-327e496d-87aa-4a70-b6f1-7317ef705845'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:55:55.733440 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-327e496d-87aa-4a70-b6f1-7317ef705845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8019c7f3ebd94943a22d342c2ba83aa6', 'request_id': 'req-327e496d-87aa-4a70-b6f1-7317ef705845'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:55:55.741453 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-327e496d-87aa-4a70-b6f1-7317ef705845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8019c7f3ebd94943a22d342c2ba83aa6', 'request_id': 'req-327e496d-87aa-4a70-b6f1-7317ef705845'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:55:55.836756 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-327e496d-87aa-4a70-b6f1-7317ef705845 admin admin] The token cache is being invalidate because project 8019c7f3ebd94943a22d342c2ba83aa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:55:55.836873 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-327e496d-87aa-4a70-b6f1-7317ef705845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:55:55.839494 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-327e496d-87aa-4a70-b6f1-7317ef705845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:55:55.839494 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1231/2443] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:55:55 2022] DELETE /identity/v3/projects/8019c7f3ebd94943a22d342c2ba83aa6 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:55:58.120031 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69b61b-f99d-4876-83c7-19e0ef0da315 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:55:58.121440 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-0e69b61b-f99d-4876-83c7-19e0ef0da315 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:55:58.132085 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:55:58.196575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69b61b-f99d-4876-83c7-19e0ef0da315 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:55:58.262303 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b5308-5fae-4b61-b97f-dd6b6ff5cd32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:55:58.262963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b5308-5fae-4b61-b97f-dd6b6ff5cd32 service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:55:58.263250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b5308-5fae-4b61-b97f-dd6b6ff5cd32 service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:55:58.263531 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b5308-5fae-4b61-b97f-dd6b6ff5cd32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:55:58.265397 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-685b5308-5fae-4b61-b97f-dd6b6ff5cd32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:55:58.323920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b5308-5fae-4b61-b97f-dd6b6ff5cd32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:55:58.325541 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685b5308-5fae-4b61-b97f-dd6b6ff5cd32 service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:55:58.335100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1213/2444] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:55:58 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:56:06.031620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=12,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:56:08.330240 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=81 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:56:19.289789 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c7393-414b-47c5-9491-8a23a48b4d30 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:19.289789 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c7393-414b-47c5-9491-8a23a48b4d30 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:19.289789 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c7393-414b-47c5-9491-8a23a48b4d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:19.307510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-294c7393-414b-47c5-9491-8a23a48b4d30 None None] MFA Rules not processed for user `c700b0adb9f54947a88c1baa86accb9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:56:19.308702 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-294c7393-414b-47c5-9491-8a23a48b4d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:56:19.362509 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1232/2445] 10.222.0.13 () {58 vars in 1009 bytes} [Fri May 27 19:56:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:56:19.374692 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b5308-5fae-4b61-b97f-dd6b6ff5cd32 service nova] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:56:19.376111 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-685b5308-5fae-4b61-b97f-dd6b6ff5cd32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:56:19.445454 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b5308-5fae-4b61-b97f-dd6b6ff5cd32 service nova] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:56:19.450434 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7bafa-f264-4d94-ba24-34d87e7b558c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37bc6d0abb454de19e1ab2f46bf4a593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:56:19.451081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7bafa-f264-4d94-ba24-34d87e7b558c service swift] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:19.451368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7bafa-f264-4d94-ba24-34d87e7b558c service swift] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:19.451650 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7bafa-f264-4d94-ba24-34d87e7b558c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:19.460552 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7bafa-f264-4d94-ba24-34d87e7b558c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:56:19.462174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7bafa-f264-4d94-ba24-34d87e7b558c service swift] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:56:19.470607 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1214/2446] 10.222.0.13 () {60 vars in 1384 bytes} [Fri May 27 19:56:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 19:56:20.381484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c7393-414b-47c5-9491-8a23a48b4d30 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:56:20.382925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-294c7393-414b-47c5-9491-8a23a48b4d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:56:20.402854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c7393-414b-47c5-9491-8a23a48b4d30 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:56:20.409639 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ab503-3e8c-4768-b1a9-c5794a9de7b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:56:20.414074 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ab503-3e8c-4768-b1a9-c5794a9de7b4 service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:20.414074 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ab503-3e8c-4768-b1a9-c5794a9de7b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:20.414074 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ab503-3e8c-4768-b1a9-c5794a9de7b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:20.414074 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-431ab503-3e8c-4768-b1a9-c5794a9de7b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:56:20.426682 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ab503-3e8c-4768-b1a9-c5794a9de7b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:56:20.428742 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ab503-3e8c-4768-b1a9-c5794a9de7b4 service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:56:20.442001 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1233/2447] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:56:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:56:28.503382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7bafa-f264-4d94-ba24-34d87e7b558c service swift] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:56:28.525295 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7bafa-f264-4d94-ba24-34d87e7b558c service swift] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:56:28.532175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191fbd9d-1e82-4b8d-b6c8-0999cf966510 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:56:28.533989 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fbd9d-1e82-4b8d-b6c8-0999cf966510 service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:28.534496 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fbd9d-1e82-4b8d-b6c8-0999cf966510 service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:28.534929 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191fbd9d-1e82-4b8d-b6c8-0999cf966510 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:28.537457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-191fbd9d-1e82-4b8d-b6c8-0999cf966510 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:56:28.551506 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191fbd9d-1e82-4b8d-b6c8-0999cf966510 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:56:28.553603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191fbd9d-1e82-4b8d-b6c8-0999cf966510 service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:56:28.567197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1215/2448] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:56:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:56:30.436326 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=31 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:56:35.346026 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ab503-3e8c-4768-b1a9-c5794a9de7b4 service nova] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:56:35.347419 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-431ab503-3e8c-4768-b1a9-c5794a9de7b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:56:35.403145 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ab503-3e8c-4768-b1a9-c5794a9de7b4 service nova] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:56:35.406755 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352639e-d7c4-4a9f-b31c-f9db1835a114 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:56:35.407583 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352639e-d7c4-4a9f-b31c-f9db1835a114 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:35.408014 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352639e-d7c4-4a9f-b31c-f9db1835a114 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:35.408296 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352639e-d7c4-4a9f-b31c-f9db1835a114 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:35.410128 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-c352639e-d7c4-4a9f-b31c-f9db1835a114 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:56:35.419400 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c352639e-d7c4-4a9f-b31c-f9db1835a114 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:56:35.420829 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c352639e-d7c4-4a9f-b31c-f9db1835a114 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:56:35.429441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1234/2449] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:56:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:56:38.562000 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=38 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:56:42.105927 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191fbd9d-1e82-4b8d-b6c8-0999cf966510 service nova] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:56:42.115570 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191fbd9d-1e82-4b8d-b6c8-0999cf966510 service nova] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:56:42.119464 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91f124b-18b9-4a1b-b382-c75581f4eb9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:56:42.120206 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f124b-18b9-4a1b-b382-c75581f4eb9a service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:42.120496 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f124b-18b9-4a1b-b382-c75581f4eb9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:42.120773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91f124b-18b9-4a1b-b382-c75581f4eb9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:42.122570 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-f91f124b-18b9-4a1b-b382-c75581f4eb9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:56:42.127222 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352639e-d7c4-4a9f-b31c-f9db1835a114 service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:56:42.140835 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c352639e-d7c4-4a9f-b31c-f9db1835a114 service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:56:42.147480 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f6de4-9faa-4da7-88df-11c9ed7198d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:56:42.148129 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f6de4-9faa-4da7-88df-11c9ed7198d5 service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:42.151572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f6de4-9faa-4da7-88df-11c9ed7198d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:42.151572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0f6de4-9faa-4da7-88df-11c9ed7198d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:42.151572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-4d0f6de4-9faa-4da7-88df-11c9ed7198d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:56:42.183894 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f124b-18b9-4a1b-b382-c75581f4eb9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:56:42.185288 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91f124b-18b9-4a1b-b382-c75581f4eb9a service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:56:42.195108 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1216/2450] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:56:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:56:42.212575 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f6de4-9faa-4da7-88df-11c9ed7198d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:56:42.214271 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0f6de4-9faa-4da7-88df-11c9ed7198d5 service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:56:42.224774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1235/2451] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:56:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:56:42.421461 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbd55c-5bc5-4832-8d0d-609172f58d01 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:42.421866 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbd55c-5bc5-4832-8d0d-609172f58d01 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:42.422357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbd55c-5bc5-4832-8d0d-609172f58d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:42.452920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-e0dbd55c-5bc5-4832-8d0d-609172f58d01 None None] MFA Rules not processed for user `c700b0adb9f54947a88c1baa86accb9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:56:42.454167 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-e0dbd55c-5bc5-4832-8d0d-609172f58d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:56:42.462504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1217/2452] 10.222.0.13 () {58 vars in 1009 bytes} [Fri May 27 19:56:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:56:42.500465 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f6de4-9faa-4da7-88df-11c9ed7198d5 service nova] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:56:42.518807 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0f6de4-9faa-4da7-88df-11c9ed7198d5 service nova] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:56:42.525728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac9fda-7b27-4a28-83ea-bb17756a33ad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37bc6d0abb454de19e1ab2f46bf4a593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:56:42.526601 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac9fda-7b27-4a28-83ea-bb17756a33ad service swift] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:42.526895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac9fda-7b27-4a28-83ea-bb17756a33ad service swift] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:42.527180 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac9fda-7b27-4a28-83ea-bb17756a33ad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:42.536159 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac9fda-7b27-4a28-83ea-bb17756a33ad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:56:42.537934 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac9fda-7b27-4a28-83ea-bb17756a33ad service swift] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:56:42.546842 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1236/2453] 10.222.0.13 () {60 vars in 1384 bytes} [Fri May 27 19:56:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 19:56:52.445839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=31 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:56:52.543634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=60 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:56:53.603943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbd55c-5bc5-4832-8d0d-609172f58d01 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:56:53.614740 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbd55c-5bc5-4832-8d0d-609172f58d01 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:56:53.620144 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12be8c3-0c2d-4748-85d5-e004786e96a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:56:53.621242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12be8c3-0c2d-4748-85d5-e004786e96a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:53.621625 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12be8c3-0c2d-4748-85d5-e004786e96a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:53.621996 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12be8c3-0c2d-4748-85d5-e004786e96a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:53.623884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-e12be8c3-0c2d-4748-85d5-e004786e96a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:56:53.693490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12be8c3-0c2d-4748-85d5-e004786e96a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:56:53.695197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12be8c3-0c2d-4748-85d5-e004786e96a7 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:56:53.710182 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1218/2454] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:56:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:56:57.868674 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac9fda-7b27-4a28-83ea-bb17756a33ad service swift] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:56:57.873083 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-7aac9fda-7b27-4a28-83ea-bb17756a33ad service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:56:57.953666 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac9fda-7b27-4a28-83ea-bb17756a33ad service swift] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:56:57.958996 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843c015-dd5b-433f-8ebd-68d42fe17ef2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2ed13a0c84f128b795002892efc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:56:57.964792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843c015-dd5b-433f-8ebd-68d42fe17ef2 service placement] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:57.964792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843c015-dd5b-433f-8ebd-68d42fe17ef2 service placement] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:57.964792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843c015-dd5b-433f-8ebd-68d42fe17ef2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:57.964792 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-a843c015-dd5b-433f-8ebd-68d42fe17ef2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:56:57.974531 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843c015-dd5b-433f-8ebd-68d42fe17ef2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:56:57.976168 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843c015-dd5b-433f-8ebd-68d42fe17ef2 service placement] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:56:57.987896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1237/2455] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:56:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:56:57.995760 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12be8c3-0c2d-4748-85d5-e004786e96a7 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:56:57.999358 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-e12be8c3-0c2d-4748-85d5-e004786e96a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:56:58.009655 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12be8c3-0c2d-4748-85d5-e004786e96a7 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:56:58.014365 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cbe524-125c-45e2-b777-10f0889f391c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:56:58.015274 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbe524-125c-45e2-b777-10f0889f391c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:58.015603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbe524-125c-45e2-b777-10f0889f391c admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:58.015845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cbe524-125c-45e2-b777-10f0889f391c admin admin] PATH_INFO: `/v3/users/a13d2c50c5204c859872e99bc45c6566` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:58.017075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cbe524-125c-45e2-b777-10f0889f391c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a13d2c50c5204c859872e99bc45c6566)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:56:58.019052 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cbe524-125c-45e2-b777-10f0889f391c admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:56:58.195679 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c6cbe524-125c-45e2-b777-10f0889f391c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a13d2c50c5204c859872e99bc45c6566'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:56:58.207321 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c6cbe524-125c-45e2-b777-10f0889f391c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a13d2c50c5204c859872e99bc45c6566'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:56:58.284946 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c6cbe524-125c-45e2-b777-10f0889f391c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a13d2c50c5204c859872e99bc45c6566'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:56:58.286703 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c6cbe524-125c-45e2-b777-10f0889f391c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a13d2c50c5204c859872e99bc45c6566'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:56:58.287269 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c6cbe524-125c-45e2-b777-10f0889f391c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a13d2c50c5204c859872e99bc45c6566'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:56:58.300475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1219/2456] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:56:57 2022] DELETE /identity/v3/users/a13d2c50c5204c859872e99bc45c6566 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:56:58.549134 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843c015-dd5b-433f-8ebd-68d42fe17ef2 service placement] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:56:58.560944 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843c015-dd5b-433f-8ebd-68d42fe17ef2 service placement] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:56:58.637108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fda372-da09-4d9e-8688-70cf353a2930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:56:58.639060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fda372-da09-4d9e-8688-70cf353a2930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:58.639479 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fda372-da09-4d9e-8688-70cf353a2930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:58.639890 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fda372-da09-4d9e-8688-70cf353a2930 admin admin] PATH_INFO: `/v3/projects/3388fd386824477385e3ad6baa4cae56` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:58.641170 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fda372-da09-4d9e-8688-70cf353a2930 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3388fd386824477385e3ad6baa4cae56)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:56:58.643245 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fda372-da09-4d9e-8688-70cf353a2930 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:56:58.787102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-14fda372-da09-4d9e-8688-70cf353a2930 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3388fd386824477385e3ad6baa4cae56', 'request_id': 'req-14fda372-da09-4d9e-8688-70cf353a2930'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:56:58.797932 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-14fda372-da09-4d9e-8688-70cf353a2930 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3388fd386824477385e3ad6baa4cae56', 'request_id': 'req-14fda372-da09-4d9e-8688-70cf353a2930'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:56:58.806175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4cc19-6050-490d-b728-04df308cc52d None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:58.806825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4cc19-6050-490d-b728-04df308cc52d None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:58.807268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4cc19-6050-490d-b728-04df308cc52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:58.809506 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-14fda372-da09-4d9e-8688-70cf353a2930 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3388fd386824477385e3ad6baa4cae56', 'request_id': 'req-14fda372-da09-4d9e-8688-70cf353a2930'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:56:58.826657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-edb4cc19-6050-490d-b728-04df308cc52d None None] MFA Rules not processed for user `c700b0adb9f54947a88c1baa86accb9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:56:58.827876 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-edb4cc19-6050-490d-b728-04df308cc52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:56:58.887789 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1220/2457] 10.222.0.13 () {58 vars in 967 bytes} [Fri May 27 19:56:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:56:58.901352 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4cc19-6050-490d-b728-04df308cc52d None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:56:58.903148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-edb4cc19-6050-490d-b728-04df308cc52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:56:58.905403 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-14fda372-da09-4d9e-8688-70cf353a2930 admin admin] The token cache is being invalidate because project 3388fd386824477385e3ad6baa4cae56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:56:58.905892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-14fda372-da09-4d9e-8688-70cf353a2930 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:56:58.906476 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-14fda372-da09-4d9e-8688-70cf353a2930 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:56:58.908123 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1238/2458] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:56:58 2022] DELETE /identity/v3/projects/3388fd386824477385e3ad6baa4cae56 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:56:58.913394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491ab61-eb18-47fc-bc0f-282644abfcc3 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:58.913846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491ab61-eb18-47fc-bc0f-282644abfcc3 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:58.914125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4491ab61-eb18-47fc-bc0f-282644abfcc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:58.917472 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:56:58.934021 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-4491ab61-eb18-47fc-bc0f-282644abfcc3 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:56:58.935271 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-4491ab61-eb18-47fc-bc0f-282644abfcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:56:58.950232 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:56:58.980756 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb4cc19-6050-490d-b728-04df308cc52d None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:56:58.986267 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781dcc6-e639-4a85-b064-48fb798a3e27 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '37bc6d0abb454de19e1ab2f46bf4a593', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:56:58.992344 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781dcc6-e639-4a85-b064-48fb798a3e27 service swift] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:58.992344 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781dcc6-e639-4a85-b064-48fb798a3e27 service swift] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:58.992344 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781dcc6-e639-4a85-b064-48fb798a3e27 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:58.992344 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-b781dcc6-e639-4a85-b064-48fb798a3e27 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:56:59.001377 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781dcc6-e639-4a85-b064-48fb798a3e27 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:56:59.002280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1239/2459] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:56:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:56:59.004013 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781dcc6-e639-4a85-b064-48fb798a3e27 service swift] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:56:59.010704 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1b33f-e792-48cd-a7d1-66bc4f9039df None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:59.016322 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1b33f-e792-48cd-a7d1-66bc4f9039df None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:59.016322 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb1b33f-e792-48cd-a7d1-66bc4f9039df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:59.016602 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1221/2460] 10.222.0.13 () {60 vars in 1384 bytes} [Fri May 27 19:56:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 19:56:59.033146 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-8bb1b33f-e792-48cd-a7d1-66bc4f9039df None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:56:59.038307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-8bb1b33f-e792-48cd-a7d1-66bc4f9039df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:56:59.050744 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1240/2461] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:56:59.055121 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781dcc6-e639-4a85-b064-48fb798a3e27 service swift] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:56:59.066828 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781dcc6-e639-4a85-b064-48fb798a3e27 service swift] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:56:59.073066 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e370be-f1de-4139-a121-ae92abeb202d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:56:59.073066 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e370be-f1de-4139-a121-ae92abeb202d admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:59.073066 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e370be-f1de-4139-a121-ae92abeb202d admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:59.073066 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e370be-f1de-4139-a121-ae92abeb202d admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:59.073629 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e370be-f1de-4139-a121-ae92abeb202d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:56:59.073935 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e370be-f1de-4139-a121-ae92abeb202d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:56:59.075534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e370be-f1de-4139-a121-ae92abeb202d admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:56:59.091734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1222/2462] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:56:59.095979 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1b33f-e792-48cd-a7d1-66bc4f9039df None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:56:59.110240 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb1b33f-e792-48cd-a7d1-66bc4f9039df None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:56:59.115436 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a440510-a762-4ddf-8cf1-cc4d7dddc238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:56:59.116108 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a440510-a762-4ddf-8cf1-cc4d7dddc238 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:59.116397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a440510-a762-4ddf-8cf1-cc4d7dddc238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:59.116679 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a440510-a762-4ddf-8cf1-cc4d7dddc238 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:59.117508 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a440510-a762-4ddf-8cf1-cc4d7dddc238 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:56:59.119452 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a440510-a762-4ddf-8cf1-cc4d7dddc238 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:56:59.195705 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1241/2463] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:56:59 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:56:59.199784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e370be-f1de-4139-a121-ae92abeb202d admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:56:59.210587 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e370be-f1de-4139-a121-ae92abeb202d admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:56:59.273097 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c608f0-88bf-4bd4-8a89-e1e4a2ddee84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:56:59.273814 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c608f0-88bf-4bd4-8a89-e1e4a2ddee84 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:59.274230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c608f0-88bf-4bd4-8a89-e1e4a2ddee84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:59.274624 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c608f0-88bf-4bd4-8a89-e1e4a2ddee84 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:59.275362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c608f0-88bf-4bd4-8a89-e1e4a2ddee84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:56:59.277506 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c608f0-88bf-4bd4-8a89-e1e4a2ddee84 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:56:59.355562 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1223/2464] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:56:59 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:56:59.361427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a440510-a762-4ddf-8cf1-cc4d7dddc238 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:56:59.372051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a440510-a762-4ddf-8cf1-cc4d7dddc238 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:56:59.377279 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4687bfc-f21d-4d5a-90f1-b09d60d0e2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:56:59.377951 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4687bfc-f21d-4d5a-90f1-b09d60d0e2f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:59.378235 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4687bfc-f21d-4d5a-90f1-b09d60d0e2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:59.378995 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4687bfc-f21d-4d5a-90f1-b09d60d0e2f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:59.379813 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4687bfc-f21d-4d5a-90f1-b09d60d0e2f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:56:59.381455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4687bfc-f21d-4d5a-90f1-b09d60d0e2f6 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:56:59.393106 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1242/2465] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:56:59.398320 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c608f0-88bf-4bd4-8a89-e1e4a2ddee84 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:56:59.408895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c608f0-88bf-4bd4-8a89-e1e4a2ddee84 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:56:59.414602 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209e6b9-2a7d-49ee-8bf4-fa763d5eb266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:56:59.415274 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209e6b9-2a7d-49ee-8bf4-fa763d5eb266 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:59.415557 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209e6b9-2a7d-49ee-8bf4-fa763d5eb266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:59.415834 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209e6b9-2a7d-49ee-8bf4-fa763d5eb266 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:59.416563 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3209e6b9-2a7d-49ee-8bf4-fa763d5eb266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:56:59.418231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3209e6b9-2a7d-49ee-8bf4-fa763d5eb266 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:56:59.429390 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1224/2466] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:56:59.433774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4687bfc-f21d-4d5a-90f1-b09d60d0e2f6 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:56:59.444597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4687bfc-f21d-4d5a-90f1-b09d60d0e2f6 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:56:59.449293 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ea507-2b5e-4fc3-a14f-734ec83c476e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:56:59.450174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ea507-2b5e-4fc3-a14f-734ec83c476e admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:59.450460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ea507-2b5e-4fc3-a14f-734ec83c476e admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:59.450744 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105ea507-2b5e-4fc3-a14f-734ec83c476e admin admin] PATH_INFO: `/v3/projects/9b8d54d76227492a82d1fb27136b1934/users/f385b1a86a43426bb50de104ddc8491c/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:59.467710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ea507-2b5e-4fc3-a14f-734ec83c476e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b8d54d76227492a82d1fb27136b1934, user_id=f385b1a86a43426bb50de104ddc8491c, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:56:59.470559 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105ea507-2b5e-4fc3-a14f-734ec83c476e admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:56:59.536973 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1243/2467] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:56:59 2022] PUT /identity/v3/projects/9b8d54d76227492a82d1fb27136b1934/users/f385b1a86a43426bb50de104ddc8491c/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:56:59.547963 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7bfa4-31db-4be6-ba36-ff5a24755714 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:59.548258 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7bfa4-31db-4be6-ba36-ff5a24755714 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:59.548539 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7bfa4-31db-4be6-ba36-ff5a24755714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:59.577960 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-0ea7bfa4-31db-4be6-ba36-ff5a24755714 None None] MFA Rules not processed for user `f385b1a86a43426bb50de104ddc8491c`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:56:59.579260 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-0ea7bfa4-31db-4be6-ba36-ff5a24755714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:56:59.635187 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:56:59.644259 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1225/2468] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:56:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:56:59.649728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ea507-2b5e-4fc3-a14f-734ec83c476e admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:56:59.715825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105ea507-2b5e-4fc3-a14f-734ec83c476e admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:56:59.720770 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85900994-ede0-48ae-b2ec-2191dab879ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:56:59.721463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85900994-ede0-48ae-b2ec-2191dab879ed admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:59.721750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85900994-ede0-48ae-b2ec-2191dab879ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:59.722095 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85900994-ede0-48ae-b2ec-2191dab879ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:59.722932 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85900994-ede0-48ae-b2ec-2191dab879ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:56:59.725620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85900994-ede0-48ae-b2ec-2191dab879ed admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:56:59.849243 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1244/2469] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:56:59 2022] POST /identity/v3/projects => generated 373 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:56:59.851716 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7bfa4-31db-4be6-ba36-ff5a24755714 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:56:59.862806 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7bfa4-31db-4be6-ba36-ff5a24755714 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:56:59.923027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e46b55-ba5a-4710-b268-0fe0de65f717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:56:59.923808 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e46b55-ba5a-4710-b268-0fe0de65f717 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:56:59.924105 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e46b55-ba5a-4710-b268-0fe0de65f717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:56:59.924376 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e46b55-ba5a-4710-b268-0fe0de65f717 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:56:59.925113 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e46b55-ba5a-4710-b268-0fe0de65f717 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:56:59.927187 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e46b55-ba5a-4710-b268-0fe0de65f717 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:00.035024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1226/2470] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:56:59 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:00.039602 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85900994-ede0-48ae-b2ec-2191dab879ed admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:00.051128 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85900994-ede0-48ae-b2ec-2191dab879ed admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:00.056593 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb2910-96f4-4142-9eee-b44e8e5d2ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:00.057294 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb2910-96f4-4142-9eee-b44e8e5d2ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:00.057583 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb2910-96f4-4142-9eee-b44e8e5d2ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:00.057893 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb2910-96f4-4142-9eee-b44e8e5d2ab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:00.058691 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb2910-96f4-4142-9eee-b44e8e5d2ab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:00.060356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb2910-96f4-4142-9eee-b44e8e5d2ab4 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:00.071062 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1245/2471] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:00.075223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e46b55-ba5a-4710-b268-0fe0de65f717 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:00.085592 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e46b55-ba5a-4710-b268-0fe0de65f717 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:00.090362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f09979e-2d63-496b-8e2b-a68f2dcd0796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:00.091246 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f09979e-2d63-496b-8e2b-a68f2dcd0796 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:00.091535 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f09979e-2d63-496b-8e2b-a68f2dcd0796 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:00.091817 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f09979e-2d63-496b-8e2b-a68f2dcd0796 admin admin] PATH_INFO: `/v3/projects/3c582658d0ad40d8a903f0d341ab70c1/users/2f9dca819e7d47b3903115956ae1973c/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:00.107420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f09979e-2d63-496b-8e2b-a68f2dcd0796 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c582658d0ad40d8a903f0d341ab70c1, user_id=2f9dca819e7d47b3903115956ae1973c, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:00.111578 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f09979e-2d63-496b-8e2b-a68f2dcd0796 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:00.228624 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1227/2472] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:00 2022] PUT /identity/v3/projects/3c582658d0ad40d8a903f0d341ab70c1/users/2f9dca819e7d47b3903115956ae1973c/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:00.232713 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb2910-96f4-4142-9eee-b44e8e5d2ab4 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:00.243135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb2910-96f4-4142-9eee-b44e8e5d2ab4 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:00.262690 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:00.300524 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f25b8-bc48-4954-b498-e1dbf7dbdd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:00.302196 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f25b8-bc48-4954-b498-e1dbf7dbdd8d admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:00.302750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f25b8-bc48-4954-b498-e1dbf7dbdd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:00.303604 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f25b8-bc48-4954-b498-e1dbf7dbdd8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:00.304394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f25b8-bc48-4954-b498-e1dbf7dbdd8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:00.306090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f25b8-bc48-4954-b498-e1dbf7dbdd8d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:00.316181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1246/2473] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:00.321437 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f09979e-2d63-496b-8e2b-a68f2dcd0796 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:00.331791 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f09979e-2d63-496b-8e2b-a68f2dcd0796 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:00.337017 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9d0a2-4bbf-4f26-9e96-03d2bf95e6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:00.337867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9d0a2-4bbf-4f26-9e96-03d2bf95e6b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:00.338154 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9d0a2-4bbf-4f26-9e96-03d2bf95e6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:00.338434 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9d0a2-4bbf-4f26-9e96-03d2bf95e6b8 admin admin] PATH_INFO: `/v3/projects/3c582658d0ad40d8a903f0d341ab70c1/users/2f9dca819e7d47b3903115956ae1973c/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:00.340389 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce9d0a2-4bbf-4f26-9e96-03d2bf95e6b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c582658d0ad40d8a903f0d341ab70c1, user_id=2f9dca819e7d47b3903115956ae1973c, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:00.343103 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce9d0a2-4bbf-4f26-9e96-03d2bf95e6b8 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:00.354540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1228/2474] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:00 2022] PUT /identity/v3/projects/3c582658d0ad40d8a903f0d341ab70c1/users/2f9dca819e7d47b3903115956ae1973c/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:00.365808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11056c65-af7b-43e3-9c75-998a46c539ca None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:00.366854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11056c65-af7b-43e3-9c75-998a46c539ca None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:00.367159 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11056c65-af7b-43e3-9c75-998a46c539ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:00.393665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-11056c65-af7b-43e3-9c75-998a46c539ca None None] MFA Rules not processed for user `2f9dca819e7d47b3903115956ae1973c`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:00.394928 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-11056c65-af7b-43e3-9c75-998a46c539ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:00.462989 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1247/2475] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:00.474360 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9d0a2-4bbf-4f26-9e96-03d2bf95e6b8 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:00.476033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-3ce9d0a2-4bbf-4f26-9e96-03d2bf95e6b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:00.532860 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9d0a2-4bbf-4f26-9e96-03d2bf95e6b8 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:00.536741 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493be870-0f87-438f-a12e-bfe0c4587d59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:00.537444 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493be870-0f87-438f-a12e-bfe0c4587d59 service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:00.537731 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493be870-0f87-438f-a12e-bfe0c4587d59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:00.538009 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493be870-0f87-438f-a12e-bfe0c4587d59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:00.546781 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493be870-0f87-438f-a12e-bfe0c4587d59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:00.548200 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493be870-0f87-438f-a12e-bfe0c4587d59 service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:00.557982 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1229/2476] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:00 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:57:01.415884 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11056c65-af7b-43e3-9c75-998a46c539ca None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:01.477869 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11056c65-af7b-43e3-9c75-998a46c539ca None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:01.482407 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b5160-4882-483b-b2f2-1697b2066900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:01.483310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b5160-4882-483b-b2f2-1697b2066900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:01.483598 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b5160-4882-483b-b2f2-1697b2066900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:01.483886 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879b5160-4882-483b-b2f2-1697b2066900 admin admin] PATH_INFO: `/v3/users/f385b1a86a43426bb50de104ddc8491c` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:01.485100 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b5160-4882-483b-b2f2-1697b2066900 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f385b1a86a43426bb50de104ddc8491c)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:01.487040 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879b5160-4882-483b-b2f2-1697b2066900 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:01.622545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:01.624037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-879b5160-4882-483b-b2f2-1697b2066900 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f385b1a86a43426bb50de104ddc8491c'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:01.636229 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-879b5160-4882-483b-b2f2-1697b2066900 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f385b1a86a43426bb50de104ddc8491c'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:01.690261 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-879b5160-4882-483b-b2f2-1697b2066900 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f385b1a86a43426bb50de104ddc8491c'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:01.690810 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-879b5160-4882-483b-b2f2-1697b2066900 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f385b1a86a43426bb50de104ddc8491c'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:01.698589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-879b5160-4882-483b-b2f2-1697b2066900 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f385b1a86a43426bb50de104ddc8491c'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:01.708614 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1248/2477] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:57:01 2022] DELETE /identity/v3/users/f385b1a86a43426bb50de104ddc8491c => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:01.710476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847bcf8-a522-4907-9119-15f246e45200 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:01.711223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847bcf8-a522-4907-9119-15f246e45200 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:01.711326 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847bcf8-a522-4907-9119-15f246e45200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:01.728936 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-e847bcf8-a522-4907-9119-15f246e45200 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:01.730958 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-e847bcf8-a522-4907-9119-15f246e45200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:01.775027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:01.788879 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1230/2478] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:01.798425 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b5160-4882-483b-b2f2-1697b2066900 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:01.799815 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-879b5160-4882-483b-b2f2-1697b2066900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:01.856136 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879b5160-4882-483b-b2f2-1697b2066900 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:01.859849 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f7e40-ee8b-4fe8-863c-c2ed7e788bc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:01.860540 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f7e40-ee8b-4fe8-863c-c2ed7e788bc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:01.860819 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f7e40-ee8b-4fe8-863c-c2ed7e788bc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:01.861134 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8f7e40-ee8b-4fe8-863c-c2ed7e788bc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:01.870874 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f7e40-ee8b-4fe8-863c-c2ed7e788bc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:01.872309 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8f7e40-ee8b-4fe8-863c-c2ed7e788bc9 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:01.882277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1249/2479] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:57:02.280934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847bcf8-a522-4907-9119-15f246e45200 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:02.282530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-e847bcf8-a522-4907-9119-15f246e45200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:02.295136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847bcf8-a522-4907-9119-15f246e45200 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:02.301390 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8ff6d-c62d-42d5-9568-094c034be8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:02.302787 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8ff6d-c62d-42d5-9568-094c034be8f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:02.303187 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8ff6d-c62d-42d5-9568-094c034be8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:02.303591 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8ff6d-c62d-42d5-9568-094c034be8f0 admin admin] PATH_INFO: `/v3/projects/9b8d54d76227492a82d1fb27136b1934` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:02.308561 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8ff6d-c62d-42d5-9568-094c034be8f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b8d54d76227492a82d1fb27136b1934)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:02.310676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8ff6d-c62d-42d5-9568-094c034be8f0 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:02.417658 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-5da8ff6d-c62d-42d5-9568-094c034be8f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b8d54d76227492a82d1fb27136b1934', 'request_id': 'req-5da8ff6d-c62d-42d5-9568-094c034be8f0'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:02.423158 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-5da8ff6d-c62d-42d5-9568-094c034be8f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b8d54d76227492a82d1fb27136b1934', 'request_id': 'req-5da8ff6d-c62d-42d5-9568-094c034be8f0'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:02.478582 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-5da8ff6d-c62d-42d5-9568-094c034be8f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b8d54d76227492a82d1fb27136b1934', 'request_id': 'req-5da8ff6d-c62d-42d5-9568-094c034be8f0'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:02.491189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-5da8ff6d-c62d-42d5-9568-094c034be8f0 admin admin] The token cache is being invalidate because project 9b8d54d76227492a82d1fb27136b1934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:57:02.491476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-5da8ff6d-c62d-42d5-9568-094c034be8f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:02.492183 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-5da8ff6d-c62d-42d5-9568-094c034be8f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:02.493926 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1231/2480] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:57:02 2022] DELETE /identity/v3/projects/9b8d54d76227492a82d1fb27136b1934 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:02.497984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f7e40-ee8b-4fe8-863c-c2ed7e788bc9 service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:02.508885 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8f7e40-ee8b-4fe8-863c-c2ed7e788bc9 service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:02.566069 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47f4d7-9e44-40ca-b1af-7b7fbcea7593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:02.566858 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47f4d7-9e44-40ca-b1af-7b7fbcea7593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:02.567733 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47f4d7-9e44-40ca-b1af-7b7fbcea7593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:02.568065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d47f4d7-9e44-40ca-b1af-7b7fbcea7593 admin admin] PATH_INFO: `/v3/users/2f9dca819e7d47b3903115956ae1973c` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:02.569394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d47f4d7-9e44-40ca-b1af-7b7fbcea7593 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f9dca819e7d47b3903115956ae1973c)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:02.571378 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d47f4d7-9e44-40ca-b1af-7b7fbcea7593 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:02.744601 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-4d47f4d7-9e44-40ca-b1af-7b7fbcea7593 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f9dca819e7d47b3903115956ae1973c'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:02.758320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-4d47f4d7-9e44-40ca-b1af-7b7fbcea7593 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f9dca819e7d47b3903115956ae1973c'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:02.760880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:02.827197 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-4d47f4d7-9e44-40ca-b1af-7b7fbcea7593 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f9dca819e7d47b3903115956ae1973c'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:02.827767 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-4d47f4d7-9e44-40ca-b1af-7b7fbcea7593 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f9dca819e7d47b3903115956ae1973c'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:02.835239 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-4d47f4d7-9e44-40ca-b1af-7b7fbcea7593 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f9dca819e7d47b3903115956ae1973c'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:02.841477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1250/2481] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:57:02 2022] DELETE /identity/v3/users/2f9dca819e7d47b3903115956ae1973c => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:03.222900 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8ff6d-c62d-42d5-9568-094c034be8f0 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:03.224848 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-5da8ff6d-c62d-42d5-9568-094c034be8f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:03.238100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8ff6d-c62d-42d5-9568-094c034be8f0 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:03.296499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da23af-11a0-4612-b65a-47ad8d174b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:03.297403 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da23af-11a0-4612-b65a-47ad8d174b56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:03.297694 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da23af-11a0-4612-b65a-47ad8d174b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:03.297975 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da23af-11a0-4612-b65a-47ad8d174b56 admin admin] PATH_INFO: `/v3/projects/3c582658d0ad40d8a903f0d341ab70c1` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:03.299154 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da23af-11a0-4612-b65a-47ad8d174b56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c582658d0ad40d8a903f0d341ab70c1)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:03.301111 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da23af-11a0-4612-b65a-47ad8d174b56 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:03.434862 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-e4da23af-11a0-4612-b65a-47ad8d174b56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c582658d0ad40d8a903f0d341ab70c1', 'request_id': 'req-e4da23af-11a0-4612-b65a-47ad8d174b56'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:03.440342 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-e4da23af-11a0-4612-b65a-47ad8d174b56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c582658d0ad40d8a903f0d341ab70c1', 'request_id': 'req-e4da23af-11a0-4612-b65a-47ad8d174b56'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:03.460453 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47f4d7-9e44-40ca-b1af-7b7fbcea7593 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:03.462126 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-4d47f4d7-9e44-40ca-b1af-7b7fbcea7593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:03.513343 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-e4da23af-11a0-4612-b65a-47ad8d174b56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c582658d0ad40d8a903f0d341ab70c1', 'request_id': 'req-e4da23af-11a0-4612-b65a-47ad8d174b56'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:03.519089 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d47f4d7-9e44-40ca-b1af-7b7fbcea7593 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:03.528339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-e4da23af-11a0-4612-b65a-47ad8d174b56 admin admin] The token cache is being invalidate because project 3c582658d0ad40d8a903f0d341ab70c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:57:03.533476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-e4da23af-11a0-4612-b65a-47ad8d174b56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:03.533476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-e4da23af-11a0-4612-b65a-47ad8d174b56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:03.533476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1232/2482] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:57:03 2022] DELETE /identity/v3/projects/3c582658d0ad40d8a903f0d341ab70c1 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:03.535654 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714094a1-db46-4255-8000-3df1cfa16f02 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:03.536070 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714094a1-db46-4255-8000-3df1cfa16f02 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:03.536351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714094a1-db46-4255-8000-3df1cfa16f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:03.543432 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:03.556162 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-714094a1-db46-4255-8000-3df1cfa16f02 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:03.557423 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-714094a1-db46-4255-8000-3df1cfa16f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:03.573384 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a16f1-178b-4830-9281-0f4c65971fe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:03.574022 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a16f1-178b-4830-9281-0f4c65971fe8 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:03.574306 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a16f1-178b-4830-9281-0f4c65971fe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:03.574585 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a16f1-178b-4830-9281-0f4c65971fe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:03.576771 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-812a16f1-178b-4830-9281-0f4c65971fe8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:03.617291 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1233/2483] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:03.625477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad1696-365d-4471-b08e-4fcd2c5b3d42 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:03.625884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad1696-365d-4471-b08e-4fcd2c5b3d42 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:03.626166 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad1696-365d-4471-b08e-4fcd2c5b3d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:03.641025 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a16f1-178b-4830-9281-0f4c65971fe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:03.642497 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a16f1-178b-4830-9281-0f4c65971fe8 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:03.644359 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-23ad1696-365d-4471-b08e-4fcd2c5b3d42 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:03.645608 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-23ad1696-365d-4471-b08e-4fcd2c5b3d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:03.652474 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1251/2484] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:57:03.653284 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1234/2485] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:57:03.657275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a16f1-178b-4830-9281-0f4c65971fe8 service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:03.667152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a16f1-178b-4830-9281-0f4c65971fe8 service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:03.671733 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bcd60f-6f41-4b21-8ded-3b554f3936cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:03.672395 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bcd60f-6f41-4b21-8ded-3b554f3936cd admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:03.672681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bcd60f-6f41-4b21-8ded-3b554f3936cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:03.672984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bcd60f-6f41-4b21-8ded-3b554f3936cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:03.673720 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bcd60f-6f41-4b21-8ded-3b554f3936cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:57:03.674031 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bcd60f-6f41-4b21-8ded-3b554f3936cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:03.675656 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bcd60f-6f41-4b21-8ded-3b554f3936cd admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:03.688206 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1252/2486] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:57:03.692502 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad1696-365d-4471-b08e-4fcd2c5b3d42 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:03.701754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad1696-365d-4471-b08e-4fcd2c5b3d42 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:03.706414 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f121ab3-b54d-4feb-8532-c5643e12431a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:03.707072 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f121ab3-b54d-4feb-8532-c5643e12431a admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:03.707358 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f121ab3-b54d-4feb-8532-c5643e12431a admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:03.707638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f121ab3-b54d-4feb-8532-c5643e12431a admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:03.708328 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f121ab3-b54d-4feb-8532-c5643e12431a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:03.710263 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f121ab3-b54d-4feb-8532-c5643e12431a admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:03.801230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1235/2487] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:57:03 2022] POST /identity/v3/projects => generated 369 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:03.806252 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bcd60f-6f41-4b21-8ded-3b554f3936cd admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:03.816877 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bcd60f-6f41-4b21-8ded-3b554f3936cd admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:03.852192 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:03.876562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23256108-81ab-4159-9e32-d5d3f4037dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:03.877197 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23256108-81ab-4159-9e32-d5d3f4037dc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:03.877533 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23256108-81ab-4159-9e32-d5d3f4037dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:03.877881 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23256108-81ab-4159-9e32-d5d3f4037dc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:03.878591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23256108-81ab-4159-9e32-d5d3f4037dc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:03.880977 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23256108-81ab-4159-9e32-d5d3f4037dc4 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:03.990370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1253/2488] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:57:03 2022] POST /identity/v3/users => generated 285 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:03.991720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f121ab3-b54d-4feb-8532-c5643e12431a admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:04.003071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f121ab3-b54d-4feb-8532-c5643e12431a admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:04.008008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53b1ec-9f99-4f19-9a07-80b7eb03b4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:04.008677 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53b1ec-9f99-4f19-9a07-80b7eb03b4ef admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:04.009144 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53b1ec-9f99-4f19-9a07-80b7eb03b4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:04.010013 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d53b1ec-9f99-4f19-9a07-80b7eb03b4ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:04.010640 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d53b1ec-9f99-4f19-9a07-80b7eb03b4ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:04.011848 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d53b1ec-9f99-4f19-9a07-80b7eb03b4ef admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:04.022814 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1236/2489] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:04.026941 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23256108-81ab-4159-9e32-d5d3f4037dc4 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:04.038169 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23256108-81ab-4159-9e32-d5d3f4037dc4 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:04.043102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ba87a-6bb6-460f-9d2a-556527d1343c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:04.043769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ba87a-6bb6-460f-9d2a-556527d1343c admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:04.044057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ba87a-6bb6-460f-9d2a-556527d1343c admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:04.044336 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ba87a-6bb6-460f-9d2a-556527d1343c admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:04.045099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ba87a-6bb6-460f-9d2a-556527d1343c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:04.046713 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ba87a-6bb6-460f-9d2a-556527d1343c admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:04.057107 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1254/2490] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:04.061157 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53b1ec-9f99-4f19-9a07-80b7eb03b4ef admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:04.072132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d53b1ec-9f99-4f19-9a07-80b7eb03b4ef admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:04.076660 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c14b0-3053-4cbf-b98e-0fd29907827c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:04.077584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c14b0-3053-4cbf-b98e-0fd29907827c admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:04.077868 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c14b0-3053-4cbf-b98e-0fd29907827c admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:04.078146 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c14b0-3053-4cbf-b98e-0fd29907827c admin admin] PATH_INFO: `/v3/projects/107cb2920f65460db5aed040ec282c02/users/66365ce4b06f4d2d8e0ca210628b42e8/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:04.094021 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23c14b0-3053-4cbf-b98e-0fd29907827c admin admin] RBAC: Authorizing `identity:create_grant(project_id=107cb2920f65460db5aed040ec282c02, user_id=66365ce4b06f4d2d8e0ca210628b42e8, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:04.096796 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23c14b0-3053-4cbf-b98e-0fd29907827c admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:04.165892 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1237/2491] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:04 2022] PUT /identity/v3/projects/107cb2920f65460db5aed040ec282c02/users/66365ce4b06f4d2d8e0ca210628b42e8/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:04.177414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549e5ba-b18a-492d-a17f-713b61a37657 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:04.177791 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549e5ba-b18a-492d-a17f-713b61a37657 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:04.178082 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1549e5ba-b18a-492d-a17f-713b61a37657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:04.205378 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-1549e5ba-b18a-492d-a17f-713b61a37657 None None] MFA Rules not processed for user `66365ce4b06f4d2d8e0ca210628b42e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:04.206605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-1549e5ba-b18a-492d-a17f-713b61a37657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:04.272874 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1255/2492] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:04 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:04.277229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c14b0-3053-4cbf-b98e-0fd29907827c admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:04.343602 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c14b0-3053-4cbf-b98e-0fd29907827c admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:04.348542 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4367281-fee4-40c0-9c1f-764bbd4c4322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:04.349245 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4367281-fee4-40c0-9c1f-764bbd4c4322 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:04.349534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4367281-fee4-40c0-9c1f-764bbd4c4322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:04.349851 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4367281-fee4-40c0-9c1f-764bbd4c4322 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:04.350557 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4367281-fee4-40c0-9c1f-764bbd4c4322 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:04.352485 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4367281-fee4-40c0-9c1f-764bbd4c4322 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:04.419630 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1238/2493] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:57:04 2022] POST /identity/v3/projects => generated 369 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:04.423232 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549e5ba-b18a-492d-a17f-713b61a37657 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:04.434781 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1549e5ba-b18a-492d-a17f-713b61a37657 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:04.492151 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd7d5c-198d-4f65-a540-8ee036bf4670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:04.492828 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd7d5c-198d-4f65-a540-8ee036bf4670 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:04.493174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd7d5c-198d-4f65-a540-8ee036bf4670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:04.493459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd7d5c-198d-4f65-a540-8ee036bf4670 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:04.494158 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd7d5c-198d-4f65-a540-8ee036bf4670 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:04.496109 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd7d5c-198d-4f65-a540-8ee036bf4670 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:04.570772 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1256/2494] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:57:04 2022] POST /identity/v3/users => generated 291 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:04.574751 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4367281-fee4-40c0-9c1f-764bbd4c4322 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:04.586665 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4367281-fee4-40c0-9c1f-764bbd4c4322 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:04.591822 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867e710-3f9f-4b30-9ded-ac30d520d691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:04.592715 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867e710-3f9f-4b30-9ded-ac30d520d691 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:04.593490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867e710-3f9f-4b30-9ded-ac30d520d691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:04.593490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867e710-3f9f-4b30-9ded-ac30d520d691 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:04.594075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9867e710-3f9f-4b30-9ded-ac30d520d691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:04.595686 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9867e710-3f9f-4b30-9ded-ac30d520d691 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:04.607822 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1239/2495] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:04.612470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd7d5c-198d-4f65-a540-8ee036bf4670 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:04.623734 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd7d5c-198d-4f65-a540-8ee036bf4670 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:04.628936 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f2144-da78-43e3-9fa3-df45b6bf2cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:04.629959 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f2144-da78-43e3-9fa3-df45b6bf2cf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:04.630246 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f2144-da78-43e3-9fa3-df45b6bf2cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:04.630848 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f2144-da78-43e3-9fa3-df45b6bf2cf8 admin admin] PATH_INFO: `/v3/projects/46501d4d124b43a4b96081bc4af9ddfc/users/bc0d357249c342dfa98c3e61edcff938/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:04.647039 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f2144-da78-43e3-9fa3-df45b6bf2cf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46501d4d124b43a4b96081bc4af9ddfc, user_id=bc0d357249c342dfa98c3e61edcff938, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:04.650098 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f2144-da78-43e3-9fa3-df45b6bf2cf8 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:04.702749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1257/2496] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:04 2022] PUT /identity/v3/projects/46501d4d124b43a4b96081bc4af9ddfc/users/bc0d357249c342dfa98c3e61edcff938/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:04.706995 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867e710-3f9f-4b30-9ded-ac30d520d691 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:04.719823 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867e710-3f9f-4b30-9ded-ac30d520d691 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:04.733730 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:04.787647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02703adc-5fba-481f-9b22-c47b5940aef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:04.788351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02703adc-5fba-481f-9b22-c47b5940aef5 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:04.788643 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02703adc-5fba-481f-9b22-c47b5940aef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:04.788924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02703adc-5fba-481f-9b22-c47b5940aef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:04.789771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02703adc-5fba-481f-9b22-c47b5940aef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:04.791518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02703adc-5fba-481f-9b22-c47b5940aef5 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:04.805047 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1240/2497] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:04 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:04.807479 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f2144-da78-43e3-9fa3-df45b6bf2cf8 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:04.818283 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f2144-da78-43e3-9fa3-df45b6bf2cf8 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:04.823880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44313f5d-809e-401a-9047-6b5afdef37c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:04.824878 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44313f5d-809e-401a-9047-6b5afdef37c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:04.825194 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44313f5d-809e-401a-9047-6b5afdef37c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:04.825480 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44313f5d-809e-401a-9047-6b5afdef37c5 admin admin] PATH_INFO: `/v3/projects/46501d4d124b43a4b96081bc4af9ddfc/users/bc0d357249c342dfa98c3e61edcff938/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:04.827683 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44313f5d-809e-401a-9047-6b5afdef37c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46501d4d124b43a4b96081bc4af9ddfc, user_id=bc0d357249c342dfa98c3e61edcff938, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:04.830456 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44313f5d-809e-401a-9047-6b5afdef37c5 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:04.837744 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:04.843561 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1258/2498] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:04 2022] PUT /identity/v3/projects/46501d4d124b43a4b96081bc4af9ddfc/users/bc0d357249c342dfa98c3e61edcff938/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:04.854874 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968302a-b3cf-4abf-8d2b-0440b3a69f06 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:04.855172 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968302a-b3cf-4abf-8d2b-0440b3a69f06 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:04.855459 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2968302a-b3cf-4abf-8d2b-0440b3a69f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:04.885115 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-2968302a-b3cf-4abf-8d2b-0440b3a69f06 None None] MFA Rules not processed for user `bc0d357249c342dfa98c3e61edcff938`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:04.886356 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-2968302a-b3cf-4abf-8d2b-0440b3a69f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:04.961807 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1241/2499] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:04 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:04.973233 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44313f5d-809e-401a-9047-6b5afdef37c5 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:04.975382 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-44313f5d-809e-401a-9047-6b5afdef37c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:05.036498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44313f5d-809e-401a-9047-6b5afdef37c5 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:05.040148 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b85e36-d1af-49ac-a90e-cb7a32ededba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:05.040797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b85e36-d1af-49ac-a90e-cb7a32ededba service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:05.041188 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b85e36-d1af-49ac-a90e-cb7a32ededba service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:05.041467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b85e36-d1af-49ac-a90e-cb7a32ededba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:05.051762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b85e36-d1af-49ac-a90e-cb7a32ededba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:05.053329 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b85e36-d1af-49ac-a90e-cb7a32ededba service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:05.063170 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1259/2500] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:04 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:57:05.169892 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968302a-b3cf-4abf-8d2b-0440b3a69f06 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:05.171354 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-2968302a-b3cf-4abf-8d2b-0440b3a69f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:05.183507 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2968302a-b3cf-4abf-8d2b-0440b3a69f06 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:05.187204 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620c803-1519-4fa5-9a34-23912a3edee9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:05.187859 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620c803-1519-4fa5-9a34-23912a3edee9 service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:05.188147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620c803-1519-4fa5-9a34-23912a3edee9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:05.188427 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620c803-1519-4fa5-9a34-23912a3edee9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:05.248012 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3620c803-1519-4fa5-9a34-23912a3edee9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:05.249415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3620c803-1519-4fa5-9a34-23912a3edee9 service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:05.258969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1242/2501] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:05 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:57:07.417482 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b85e36-d1af-49ac-a90e-cb7a32ededba service cinder] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:07.419501 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-44b85e36-d1af-49ac-a90e-cb7a32ededba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:07.494052 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b85e36-d1af-49ac-a90e-cb7a32ededba service cinder] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:07.498841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b5502-1ae9-48f1-a8e7-18e970b23ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:07.499734 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b5502-1ae9-48f1-a8e7-18e970b23ebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:07.499998 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b5502-1ae9-48f1-a8e7-18e970b23ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:07.500272 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b5502-1ae9-48f1-a8e7-18e970b23ebd admin admin] PATH_INFO: `/v3/users/3c9aa6527c13466bbe5dce68a3b81ab5` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:07.501479 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b5502-1ae9-48f1-a8e7-18e970b23ebd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c9aa6527c13466bbe5dce68a3b81ab5)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:07.503412 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b5502-1ae9-48f1-a8e7-18e970b23ebd admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:07.720337 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-817b5502-1ae9-48f1-a8e7-18e970b23ebd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c9aa6527c13466bbe5dce68a3b81ab5'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:07.733683 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-817b5502-1ae9-48f1-a8e7-18e970b23ebd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c9aa6527c13466bbe5dce68a3b81ab5'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:07.804298 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-817b5502-1ae9-48f1-a8e7-18e970b23ebd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c9aa6527c13466bbe5dce68a3b81ab5'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:07.804983 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-817b5502-1ae9-48f1-a8e7-18e970b23ebd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c9aa6527c13466bbe5dce68a3b81ab5'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:07.812499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-817b5502-1ae9-48f1-a8e7-18e970b23ebd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c9aa6527c13466bbe5dce68a3b81ab5'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:07.818811 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1260/2502] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:57:07 2022] DELETE /identity/v3/users/3c9aa6527c13466bbe5dce68a3b81ab5 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:08.080174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620c803-1519-4fa5-9a34-23912a3edee9 service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:08.095303 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620c803-1519-4fa5-9a34-23912a3edee9 service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:08.134244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:08.158682 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d596ef5-0bba-42c4-9c45-3801c061b136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:08.159558 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d596ef5-0bba-42c4-9c45-3801c061b136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:08.159872 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d596ef5-0bba-42c4-9c45-3801c061b136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:08.160267 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d596ef5-0bba-42c4-9c45-3801c061b136 admin admin] PATH_INFO: `/v3/projects/c994c57a63464397bd567b97dbc3097e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:08.162131 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d596ef5-0bba-42c4-9c45-3801c061b136 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c994c57a63464397bd567b97dbc3097e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:08.164129 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d596ef5-0bba-42c4-9c45-3801c061b136 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:08.290621 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-3d596ef5-0bba-42c4-9c45-3801c061b136 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c994c57a63464397bd567b97dbc3097e', 'request_id': 'req-3d596ef5-0bba-42c4-9c45-3801c061b136'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:08.296459 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-3d596ef5-0bba-42c4-9c45-3801c061b136 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c994c57a63464397bd567b97dbc3097e', 'request_id': 'req-3d596ef5-0bba-42c4-9c45-3801c061b136'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:08.387242 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-3d596ef5-0bba-42c4-9c45-3801c061b136 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c994c57a63464397bd567b97dbc3097e', 'request_id': 'req-3d596ef5-0bba-42c4-9c45-3801c061b136'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:08.400283 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-3d596ef5-0bba-42c4-9c45-3801c061b136 admin admin] The token cache is being invalidate because project c994c57a63464397bd567b97dbc3097e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:57:08.400573 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-3d596ef5-0bba-42c4-9c45-3801c061b136 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:08.401233 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-3d596ef5-0bba-42c4-9c45-3801c061b136 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:08.402858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1243/2503] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:57:08 2022] DELETE /identity/v3/projects/c994c57a63464397bd567b97dbc3097e => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:08.407728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eed87-a438-4c1c-87b1-5cac8777c394 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:08.408055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eed87-a438-4c1c-87b1-5cac8777c394 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:08.408401 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790eed87-a438-4c1c-87b1-5cac8777c394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:08.428149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-790eed87-a438-4c1c-87b1-5cac8777c394 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:08.429397 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-790eed87-a438-4c1c-87b1-5cac8777c394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:08.443394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:08.491470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1261/2504] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:08.499138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254d6b6-8d76-4b8e-843b-19cfeb3b8835 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:08.499437 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254d6b6-8d76-4b8e-843b-19cfeb3b8835 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:08.499724 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c254d6b6-8d76-4b8e-843b-19cfeb3b8835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:08.520971 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-c254d6b6-8d76-4b8e-843b-19cfeb3b8835 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:08.525410 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-c254d6b6-8d76-4b8e-843b-19cfeb3b8835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:08.530836 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1244/2505] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:08.535105 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790eed87-a438-4c1c-87b1-5cac8777c394 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:08.545107 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790eed87-a438-4c1c-87b1-5cac8777c394 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:08.549757 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e213909-5a32-4e57-b4e5-a09e74ca687e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:08.550416 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e213909-5a32-4e57-b4e5-a09e74ca687e admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:08.550704 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e213909-5a32-4e57-b4e5-a09e74ca687e admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:08.551548 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e213909-5a32-4e57-b4e5-a09e74ca687e admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:08.552256 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e213909-5a32-4e57-b4e5-a09e74ca687e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:57:08.552566 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e213909-5a32-4e57-b4e5-a09e74ca687e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:08.554220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e213909-5a32-4e57-b4e5-a09e74ca687e admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:08.565578 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1262/2506] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:57:08.570325 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254d6b6-8d76-4b8e-843b-19cfeb3b8835 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:08.579462 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c254d6b6-8d76-4b8e-843b-19cfeb3b8835 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:08.584315 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68ca27-80d5-4f41-a66f-481034119bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:08.584997 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68ca27-80d5-4f41-a66f-481034119bda admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:08.585290 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68ca27-80d5-4f41-a66f-481034119bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:08.585570 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e68ca27-80d5-4f41-a66f-481034119bda admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:08.586521 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e68ca27-80d5-4f41-a66f-481034119bda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:08.588424 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e68ca27-80d5-4f41-a66f-481034119bda admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:08.697713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1245/2507] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:57:08 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:08.701867 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e213909-5a32-4e57-b4e5-a09e74ca687e admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:08.711953 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e213909-5a32-4e57-b4e5-a09e74ca687e admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:08.766375 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0b52b-aa70-4a34-b141-231e73651d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:08.767033 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0b52b-aa70-4a34-b141-231e73651d5f admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:08.767316 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0b52b-aa70-4a34-b141-231e73651d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:08.767592 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0b52b-aa70-4a34-b141-231e73651d5f admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:08.768276 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0b52b-aa70-4a34-b141-231e73651d5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:08.770352 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0b52b-aa70-4a34-b141-231e73651d5f admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:08.868458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1263/2508] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:57:08 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:08.872570 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68ca27-80d5-4f41-a66f-481034119bda admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:08.883505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e68ca27-80d5-4f41-a66f-481034119bda admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:08.888293 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb28d73-6681-44e0-910d-3de4bb7b83c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:08.889178 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb28d73-6681-44e0-910d-3de4bb7b83c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:08.889464 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb28d73-6681-44e0-910d-3de4bb7b83c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:08.889769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb28d73-6681-44e0-910d-3de4bb7b83c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:08.890513 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb28d73-6681-44e0-910d-3de4bb7b83c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:08.892149 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb28d73-6681-44e0-910d-3de4bb7b83c0 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:08.902786 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1246/2509] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:08.906930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0b52b-aa70-4a34-b141-231e73651d5f admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:08.917865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0b52b-aa70-4a34-b141-231e73651d5f admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:08.922824 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8cccd-07dd-4cb7-9c21-174a7fb7f77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:08.923554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8cccd-07dd-4cb7-9c21-174a7fb7f77b admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:08.923858 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8cccd-07dd-4cb7-9c21-174a7fb7f77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:08.924142 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8cccd-07dd-4cb7-9c21-174a7fb7f77b admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:08.924866 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8cccd-07dd-4cb7-9c21-174a7fb7f77b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:08.927253 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8cccd-07dd-4cb7-9c21-174a7fb7f77b admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:08.937825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1264/2510] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:08.942059 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb28d73-6681-44e0-910d-3de4bb7b83c0 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:08.952288 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb28d73-6681-44e0-910d-3de4bb7b83c0 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:08.956981 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc960dc6-1669-4f1e-a224-9a1f80481b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:08.958096 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc960dc6-1669-4f1e-a224-9a1f80481b91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:08.958388 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc960dc6-1669-4f1e-a224-9a1f80481b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:08.958668 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc960dc6-1669-4f1e-a224-9a1f80481b91 admin admin] PATH_INFO: `/v3/projects/517b8862a76443f09cbee095c54cd4ea/users/d8c613270c654a21aba84643f4844756/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:08.973961 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc960dc6-1669-4f1e-a224-9a1f80481b91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=517b8862a76443f09cbee095c54cd4ea, user_id=d8c613270c654a21aba84643f4844756, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:08.976617 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc960dc6-1669-4f1e-a224-9a1f80481b91 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:09.026622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1247/2511] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:08 2022] PUT /identity/v3/projects/517b8862a76443f09cbee095c54cd4ea/users/d8c613270c654a21aba84643f4844756/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:09.037767 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e2cab-9830-4120-9199-45ea25531b6d None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:09.038214 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e2cab-9830-4120-9199-45ea25531b6d None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:09.038632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e2cab-9830-4120-9199-45ea25531b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:09.067090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-969e2cab-9830-4120-9199-45ea25531b6d None None] MFA Rules not processed for user `d8c613270c654a21aba84643f4844756`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:09.068297 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-969e2cab-9830-4120-9199-45ea25531b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:09.133389 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1265/2512] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:09.144032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc960dc6-1669-4f1e-a224-9a1f80481b91 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:09.145847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-dc960dc6-1669-4f1e-a224-9a1f80481b91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:09.204563 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:09.220584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc960dc6-1669-4f1e-a224-9a1f80481b91 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:09.226458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9ccbc-b12f-4671-a9cf-54a09eb1fcba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:09.227109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9ccbc-b12f-4671-a9cf-54a09eb1fcba service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:09.227396 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9ccbc-b12f-4671-a9cf-54a09eb1fcba service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:09.227676 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9ccbc-b12f-4671-a9cf-54a09eb1fcba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:09.235985 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9ccbc-b12f-4671-a9cf-54a09eb1fcba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:09.237651 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9ccbc-b12f-4671-a9cf-54a09eb1fcba service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:09.250610 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1248/2513] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:57:10.298492 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e2cab-9830-4120-9199-45ea25531b6d None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:10.299963 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-969e2cab-9830-4120-9199-45ea25531b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:10.318514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:10.364590 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e2cab-9830-4120-9199-45ea25531b6d None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:10.369270 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b5ea6-4fa5-4a2d-9301-8e4aa66926b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:10.370394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b5ea6-4fa5-4a2d-9301-8e4aa66926b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:10.371097 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b5ea6-4fa5-4a2d-9301-8e4aa66926b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:10.371381 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b5ea6-4fa5-4a2d-9301-8e4aa66926b7 admin admin] PATH_INFO: `/v3/users/66365ce4b06f4d2d8e0ca210628b42e8` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:10.372616 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b5ea6-4fa5-4a2d-9301-8e4aa66926b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66365ce4b06f4d2d8e0ca210628b42e8)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:10.374596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b5ea6-4fa5-4a2d-9301-8e4aa66926b7 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:10.636299 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-4d6b5ea6-4fa5-4a2d-9301-8e4aa66926b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66365ce4b06f4d2d8e0ca210628b42e8'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:10.649660 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-4d6b5ea6-4fa5-4a2d-9301-8e4aa66926b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66365ce4b06f4d2d8e0ca210628b42e8'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:10.735881 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-4d6b5ea6-4fa5-4a2d-9301-8e4aa66926b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66365ce4b06f4d2d8e0ca210628b42e8'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:10.736500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-4d6b5ea6-4fa5-4a2d-9301-8e4aa66926b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66365ce4b06f4d2d8e0ca210628b42e8'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:10.746880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-4d6b5ea6-4fa5-4a2d-9301-8e4aa66926b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66365ce4b06f4d2d8e0ca210628b42e8'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:10.753256 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1266/2514] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:57:10 2022] DELETE /identity/v3/users/66365ce4b06f4d2d8e0ca210628b42e8 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:10.758838 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbeba84-e919-4282-90fb-080bdae16a23 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:10.759369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbeba84-e919-4282-90fb-080bdae16a23 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:10.759369 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbeba84-e919-4282-90fb-080bdae16a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:10.776279 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-bbbeba84-e919-4282-90fb-080bdae16a23 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:10.777435 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-bbbeba84-e919-4282-90fb-080bdae16a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:10.838702 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1249/2515] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:10.851661 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b5ea6-4fa5-4a2d-9301-8e4aa66926b7 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:10.853149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-4d6b5ea6-4fa5-4a2d-9301-8e4aa66926b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:10.912537 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b5ea6-4fa5-4a2d-9301-8e4aa66926b7 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:10.916110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5dbcf5-2754-43bb-9859-7c09d8c77d50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:10.916749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5dbcf5-2754-43bb-9859-7c09d8c77d50 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:10.917148 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5dbcf5-2754-43bb-9859-7c09d8c77d50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:10.917426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5dbcf5-2754-43bb-9859-7c09d8c77d50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:10.927021 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5dbcf5-2754-43bb-9859-7c09d8c77d50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:10.928451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5dbcf5-2754-43bb-9859-7c09d8c77d50 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:10.938146 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1267/2516] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:57:11.350373 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbeba84-e919-4282-90fb-080bdae16a23 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:11.351806 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-bbbeba84-e919-4282-90fb-080bdae16a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:11.364403 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbeba84-e919-4282-90fb-080bdae16a23 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:11.369074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e8812-528b-4dbd-9be1-2d7221353ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:11.369943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e8812-528b-4dbd-9be1-2d7221353ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:11.370225 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e8812-528b-4dbd-9be1-2d7221353ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:11.370496 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e8812-528b-4dbd-9be1-2d7221353ecc admin admin] PATH_INFO: `/v3/projects/107cb2920f65460db5aed040ec282c02` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:11.371692 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e8812-528b-4dbd-9be1-2d7221353ecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=107cb2920f65460db5aed040ec282c02)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:11.373693 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e8812-528b-4dbd-9be1-2d7221353ecc admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:11.509212 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c40e8812-528b-4dbd-9be1-2d7221353ecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '107cb2920f65460db5aed040ec282c02', 'request_id': 'req-c40e8812-528b-4dbd-9be1-2d7221353ecc'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:11.515454 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c40e8812-528b-4dbd-9be1-2d7221353ecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '107cb2920f65460db5aed040ec282c02', 'request_id': 'req-c40e8812-528b-4dbd-9be1-2d7221353ecc'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:11.638682 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c40e8812-528b-4dbd-9be1-2d7221353ecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '107cb2920f65460db5aed040ec282c02', 'request_id': 'req-c40e8812-528b-4dbd-9be1-2d7221353ecc'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:11.651112 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c40e8812-528b-4dbd-9be1-2d7221353ecc admin admin] The token cache is being invalidate because project 107cb2920f65460db5aed040ec282c02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:57:11.651403 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c40e8812-528b-4dbd-9be1-2d7221353ecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:11.652117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c40e8812-528b-4dbd-9be1-2d7221353ecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:11.653734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1250/2517] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:57:11 2022] DELETE /identity/v3/projects/107cb2920f65460db5aed040ec282c02 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:11.658078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5dbcf5-2754-43bb-9859-7c09d8c77d50 service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:11.668090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5dbcf5-2754-43bb-9859-7c09d8c77d50 service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:11.701261 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:11.724766 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42edc573-8024-4fe2-a801-a11c94f37909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:11.725701 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edc573-8024-4fe2-a801-a11c94f37909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:11.725984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edc573-8024-4fe2-a801-a11c94f37909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:11.726283 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edc573-8024-4fe2-a801-a11c94f37909 admin admin] PATH_INFO: `/v3/users/bc0d357249c342dfa98c3e61edcff938` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:11.727444 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42edc573-8024-4fe2-a801-a11c94f37909 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc0d357249c342dfa98c3e61edcff938)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:11.729408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42edc573-8024-4fe2-a801-a11c94f37909 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:11.893136 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-42edc573-8024-4fe2-a801-a11c94f37909 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc0d357249c342dfa98c3e61edcff938'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:11.905534 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-42edc573-8024-4fe2-a801-a11c94f37909 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc0d357249c342dfa98c3e61edcff938'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:12.015319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-42edc573-8024-4fe2-a801-a11c94f37909 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc0d357249c342dfa98c3e61edcff938'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:12.015927 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-42edc573-8024-4fe2-a801-a11c94f37909 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc0d357249c342dfa98c3e61edcff938'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:12.024138 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-42edc573-8024-4fe2-a801-a11c94f37909 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc0d357249c342dfa98c3e61edcff938'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:12.030188 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1268/2518] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:57:11 2022] DELETE /identity/v3/users/bc0d357249c342dfa98c3e61edcff938 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:12.434433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e8812-528b-4dbd-9be1-2d7221353ecc admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:12.435894 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-c40e8812-528b-4dbd-9be1-2d7221353ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:12.449669 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e8812-528b-4dbd-9be1-2d7221353ecc admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:12.507049 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275d843-8798-41df-ac4d-ad40e44e2b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:12.507921 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275d843-8798-41df-ac4d-ad40e44e2b16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:12.508203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275d843-8798-41df-ac4d-ad40e44e2b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:12.508478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275d843-8798-41df-ac4d-ad40e44e2b16 admin admin] PATH_INFO: `/v3/projects/46501d4d124b43a4b96081bc4af9ddfc` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:12.509815 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275d843-8798-41df-ac4d-ad40e44e2b16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46501d4d124b43a4b96081bc4af9ddfc)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:12.511720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3275d843-8798-41df-ac4d-ad40e44e2b16 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:12.524898 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:12.587174 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-3275d843-8798-41df-ac4d-ad40e44e2b16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46501d4d124b43a4b96081bc4af9ddfc', 'request_id': 'req-3275d843-8798-41df-ac4d-ad40e44e2b16'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:12.593725 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-3275d843-8798-41df-ac4d-ad40e44e2b16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46501d4d124b43a4b96081bc4af9ddfc', 'request_id': 'req-3275d843-8798-41df-ac4d-ad40e44e2b16'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:12.626340 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-3275d843-8798-41df-ac4d-ad40e44e2b16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46501d4d124b43a4b96081bc4af9ddfc', 'request_id': 'req-3275d843-8798-41df-ac4d-ad40e44e2b16'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:12.638753 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-3275d843-8798-41df-ac4d-ad40e44e2b16 admin admin] The token cache is being invalidate because project 46501d4d124b43a4b96081bc4af9ddfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:57:12.639042 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-3275d843-8798-41df-ac4d-ad40e44e2b16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:12.639677 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-3275d843-8798-41df-ac4d-ad40e44e2b16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:12.641264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1251/2519] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:57:12 2022] DELETE /identity/v3/projects/46501d4d124b43a4b96081bc4af9ddfc => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:12.646931 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae5747-3810-4d3a-afd1-001549551f7d None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:12.647226 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae5747-3810-4d3a-afd1-001549551f7d None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:12.647509 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ae5747-3810-4d3a-afd1-001549551f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:12.665915 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-14ae5747-3810-4d3a-afd1-001549551f7d None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:12.667093 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-14ae5747-3810-4d3a-afd1-001549551f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:12.726109 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1269/2520] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:57:12.734750 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785a89f-3aa8-4c9f-993b-70bfe7169490 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:12.735043 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785a89f-3aa8-4c9f-993b-70bfe7169490 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:12.735328 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785a89f-3aa8-4c9f-993b-70bfe7169490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:12.753427 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-a785a89f-3aa8-4c9f-993b-70bfe7169490 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:12.755359 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-a785a89f-3aa8-4c9f-993b-70bfe7169490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:12.763159 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1252/2521] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:12.767194 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae5747-3810-4d3a-afd1-001549551f7d None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:12.777570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ae5747-3810-4d3a-afd1-001549551f7d None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:12.782115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e3650-efe2-466c-b4c8-89e902b5c687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:12.782761 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e3650-efe2-466c-b4c8-89e902b5c687 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:12.783044 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e3650-efe2-466c-b4c8-89e902b5c687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:12.783319 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46e3650-efe2-466c-b4c8-89e902b5c687 admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:12.784036 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e3650-efe2-466c-b4c8-89e902b5c687 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:57:12.784339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e3650-efe2-466c-b4c8-89e902b5c687 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:12.786070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46e3650-efe2-466c-b4c8-89e902b5c687 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:12.797476 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1270/2522] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:57:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:57:12.801796 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785a89f-3aa8-4c9f-993b-70bfe7169490 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:12.810713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785a89f-3aa8-4c9f-993b-70bfe7169490 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:12.815274 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62253e-0e89-4fd7-aada-21a6956f407c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:12.815913 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62253e-0e89-4fd7-aada-21a6956f407c admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:12.816197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62253e-0e89-4fd7-aada-21a6956f407c admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:12.816473 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c62253e-0e89-4fd7-aada-21a6956f407c admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:12.817954 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62253e-0e89-4fd7-aada-21a6956f407c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:12.819844 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c62253e-0e89-4fd7-aada-21a6956f407c admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:12.897272 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1253/2523] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:57:12 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:57:12.901490 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e3650-efe2-466c-b4c8-89e902b5c687 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:12.912202 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46e3650-efe2-466c-b4c8-89e902b5c687 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:12.956355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:12.971903 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc6ad7-9508-43a5-a004-c643bff74608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:12.972576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc6ad7-9508-43a5-a004-c643bff74608 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:12.972895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc6ad7-9508-43a5-a004-c643bff74608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:12.973485 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc6ad7-9508-43a5-a004-c643bff74608 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:12.974291 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efc6ad7-9508-43a5-a004-c643bff74608 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:12.976742 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efc6ad7-9508-43a5-a004-c643bff74608 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:13.050506 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1271/2524] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:57:12 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:13.054863 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62253e-0e89-4fd7-aada-21a6956f407c admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:13.065055 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c62253e-0e89-4fd7-aada-21a6956f407c admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:13.071548 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4224b4d2-031c-47dd-93b1-16bda030d93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:13.072228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224b4d2-031c-47dd-93b1-16bda030d93a admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:13.072517 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224b4d2-031c-47dd-93b1-16bda030d93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:13.072800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4224b4d2-031c-47dd-93b1-16bda030d93a admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:13.073568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4224b4d2-031c-47dd-93b1-16bda030d93a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:13.075169 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4224b4d2-031c-47dd-93b1-16bda030d93a admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:13.086323 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1254/2525] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:13.090300 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc6ad7-9508-43a5-a004-c643bff74608 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:13.100165 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc6ad7-9508-43a5-a004-c643bff74608 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:13.104790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7de4a-0469-4af8-adc6-9f2b421adfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:13.105485 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7de4a-0469-4af8-adc6-9f2b421adfb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:13.108899 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7de4a-0469-4af8-adc6-9f2b421adfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:13.108899 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7de4a-0469-4af8-adc6-9f2b421adfb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:13.108899 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7de4a-0469-4af8-adc6-9f2b421adfb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:13.108899 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7de4a-0469-4af8-adc6-9f2b421adfb7 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:13.118817 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1272/2526] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:57:13.122984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4224b4d2-031c-47dd-93b1-16bda030d93a admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:13.133635 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4224b4d2-031c-47dd-93b1-16bda030d93a admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:13.138357 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734cf63-f5cf-4745-b8d3-0402d3a6cef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:13.139233 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734cf63-f5cf-4745-b8d3-0402d3a6cef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:13.139517 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734cf63-f5cf-4745-b8d3-0402d3a6cef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:13.139800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3734cf63-f5cf-4745-b8d3-0402d3a6cef0 admin admin] PATH_INFO: `/v3/projects/48aa5eceffe7496aa7d662acffbae98b/users/0b1c961e895b46fa98c6a00a34cad01c/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:13.154308 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734cf63-f5cf-4745-b8d3-0402d3a6cef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48aa5eceffe7496aa7d662acffbae98b, user_id=0b1c961e895b46fa98c6a00a34cad01c, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:13.157033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3734cf63-f5cf-4745-b8d3-0402d3a6cef0 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:13.221205 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1255/2527] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:13 2022] PUT /identity/v3/projects/48aa5eceffe7496aa7d662acffbae98b/users/0b1c961e895b46fa98c6a00a34cad01c/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:13.231549 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403ed4c-0984-4e5e-b129-f6375f05e1ed None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:13.231847 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403ed4c-0984-4e5e-b129-f6375f05e1ed None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:13.232132 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403ed4c-0984-4e5e-b129-f6375f05e1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:13.259166 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-e403ed4c-0984-4e5e-b129-f6375f05e1ed None None] MFA Rules not processed for user `0b1c961e895b46fa98c6a00a34cad01c`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:13.260365 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-e403ed4c-0984-4e5e-b129-f6375f05e1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:13.325443 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1273/2528] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:13.330088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734cf63-f5cf-4745-b8d3-0402d3a6cef0 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:13.390148 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3734cf63-f5cf-4745-b8d3-0402d3a6cef0 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:13.395264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27f32c-91bf-4315-b5c7-9a22c2301f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:13.395923 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27f32c-91bf-4315-b5c7-9a22c2301f11 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:13.396208 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27f32c-91bf-4315-b5c7-9a22c2301f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:13.396490 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f27f32c-91bf-4315-b5c7-9a22c2301f11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:13.397264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f27f32c-91bf-4315-b5c7-9a22c2301f11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:13.399214 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f27f32c-91bf-4315-b5c7-9a22c2301f11 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:13.497481 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1256/2529] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:57:13 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:57:13.501495 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403ed4c-0984-4e5e-b129-f6375f05e1ed None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:13.512835 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403ed4c-0984-4e5e-b129-f6375f05e1ed None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:13.567946 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f3c99-76f3-4fff-a97d-72e67fec6b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:13.568612 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f3c99-76f3-4fff-a97d-72e67fec6b9a admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:13.568897 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f3c99-76f3-4fff-a97d-72e67fec6b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:13.569207 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f3c99-76f3-4fff-a97d-72e67fec6b9a admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:13.569897 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f3c99-76f3-4fff-a97d-72e67fec6b9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:13.571824 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1f3c99-76f3-4fff-a97d-72e67fec6b9a admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:13.650360 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1274/2530] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:57:13 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:13.654328 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27f32c-91bf-4315-b5c7-9a22c2301f11 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:13.665138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f27f32c-91bf-4315-b5c7-9a22c2301f11 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:13.669711 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f0b58-d31e-471f-8f21-3e106f87915d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:13.670374 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f0b58-d31e-471f-8f21-3e106f87915d admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:13.670661 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f0b58-d31e-471f-8f21-3e106f87915d admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:13.670967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f0b58-d31e-471f-8f21-3e106f87915d admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:13.671694 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2f0b58-d31e-471f-8f21-3e106f87915d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:13.673344 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2f0b58-d31e-471f-8f21-3e106f87915d admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:13.689901 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1257/2531] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:13.694179 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f3c99-76f3-4fff-a97d-72e67fec6b9a admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:13.704580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f3c99-76f3-4fff-a97d-72e67fec6b9a admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:13.709390 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978fd7f7-2efe-4950-9cbb-f41150810809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:13.710121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fd7f7-2efe-4950-9cbb-f41150810809 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:13.710406 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fd7f7-2efe-4950-9cbb-f41150810809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:13.710962 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978fd7f7-2efe-4950-9cbb-f41150810809 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:13.711687 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978fd7f7-2efe-4950-9cbb-f41150810809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:13.713394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978fd7f7-2efe-4950-9cbb-f41150810809 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:13.724038 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1275/2532] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:13.728103 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f0b58-d31e-471f-8f21-3e106f87915d admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:13.737745 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f0b58-d31e-471f-8f21-3e106f87915d admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:13.743189 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d43abb-e07c-478f-b946-429f9232321c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:13.744076 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d43abb-e07c-478f-b946-429f9232321c admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:13.744360 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d43abb-e07c-478f-b946-429f9232321c admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:13.744636 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d43abb-e07c-478f-b946-429f9232321c admin admin] PATH_INFO: `/v3/projects/fd7fcd9619d246bcba766a53d4022c64/users/6143e2076f1e4309893e7878c6171be1/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:13.752516 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:13.759686 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d43abb-e07c-478f-b946-429f9232321c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd7fcd9619d246bcba766a53d4022c64, user_id=6143e2076f1e4309893e7878c6171be1, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:13.762481 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d43abb-e07c-478f-b946-429f9232321c admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:13.885456 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1258/2533] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:13 2022] PUT /identity/v3/projects/fd7fcd9619d246bcba766a53d4022c64/users/6143e2076f1e4309893e7878c6171be1/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:13.895743 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84ed37-27c9-4d72-80b3-763811d6fec2 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:13.896033 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84ed37-27c9-4d72-80b3-763811d6fec2 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:13.896399 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce84ed37-27c9-4d72-80b3-763811d6fec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:13.921779 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-ce84ed37-27c9-4d72-80b3-763811d6fec2 None None] MFA Rules not processed for user `6143e2076f1e4309893e7878c6171be1`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:13.923018 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ce84ed37-27c9-4d72-80b3-763811d6fec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:13.943026 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:13.986669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1276/2534] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:13.991763 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d43abb-e07c-478f-b946-429f9232321c admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:14.056811 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d43abb-e07c-478f-b946-429f9232321c admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:14.061543 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0658f5d-e3e5-4f77-a4d7-45306346f593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:14.062207 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0658f5d-e3e5-4f77-a4d7-45306346f593 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:14.062494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0658f5d-e3e5-4f77-a4d7-45306346f593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:14.062772 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0658f5d-e3e5-4f77-a4d7-45306346f593 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:14.063473 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0658f5d-e3e5-4f77-a4d7-45306346f593 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:14.065480 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0658f5d-e3e5-4f77-a4d7-45306346f593 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:14.143236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1259/2535] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:57:13 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:14.147448 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84ed37-27c9-4d72-80b3-763811d6fec2 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:14.158115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce84ed37-27c9-4d72-80b3-763811d6fec2 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:14.213461 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ae668-90f1-41ee-9997-d12d4cedcbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:14.214127 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ae668-90f1-41ee-9997-d12d4cedcbcc admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:14.214413 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ae668-90f1-41ee-9997-d12d4cedcbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:14.214693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0ae668-90f1-41ee-9997-d12d4cedcbcc admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:14.215371 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0ae668-90f1-41ee-9997-d12d4cedcbcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:14.217341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0ae668-90f1-41ee-9997-d12d4cedcbcc admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:14.278039 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1277/2536] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:57:14 2022] POST /identity/v3/users => generated 299 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:14.281988 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0658f5d-e3e5-4f77-a4d7-45306346f593 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:14.292319 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0658f5d-e3e5-4f77-a4d7-45306346f593 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:14.296887 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91779efc-f850-4b95-874a-5b900668af58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:14.297593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91779efc-f850-4b95-874a-5b900668af58 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:14.297875 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91779efc-f850-4b95-874a-5b900668af58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:14.298153 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91779efc-f850-4b95-874a-5b900668af58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:14.298880 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91779efc-f850-4b95-874a-5b900668af58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:14.300477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91779efc-f850-4b95-874a-5b900668af58 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:14.311287 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1260/2537] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:14.315384 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ae668-90f1-41ee-9997-d12d4cedcbcc admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:14.326168 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0ae668-90f1-41ee-9997-d12d4cedcbcc admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:14.331075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f57666-528a-488d-826a-cc58282010f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:14.336135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f57666-528a-488d-826a-cc58282010f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:14.336135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f57666-528a-488d-826a-cc58282010f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:14.336135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f57666-528a-488d-826a-cc58282010f5 admin admin] PATH_INFO: `/v3/projects/2ceeb02ecabb4d5c810819c86e28eb5c/users/4e6b8d07af354ed8866215fb2a217679/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:14.347962 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f57666-528a-488d-826a-cc58282010f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ceeb02ecabb4d5c810819c86e28eb5c, user_id=4e6b8d07af354ed8866215fb2a217679, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:14.350665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f57666-528a-488d-826a-cc58282010f5 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:14.403787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1278/2538] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:14 2022] PUT /identity/v3/projects/2ceeb02ecabb4d5c810819c86e28eb5c/users/4e6b8d07af354ed8866215fb2a217679/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:14.407854 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91779efc-f850-4b95-874a-5b900668af58 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:14.418034 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91779efc-f850-4b95-874a-5b900668af58 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:14.474316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c75c6b-3e72-4638-9ffe-dbb135bd3e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:14.474984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c75c6b-3e72-4638-9ffe-dbb135bd3e0e admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:14.475272 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c75c6b-3e72-4638-9ffe-dbb135bd3e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:14.475554 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c75c6b-3e72-4638-9ffe-dbb135bd3e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:14.476273 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c75c6b-3e72-4638-9ffe-dbb135bd3e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:14.477907 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c75c6b-3e72-4638-9ffe-dbb135bd3e0e admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:14.487568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1261/2539] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:14.492500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f57666-528a-488d-826a-cc58282010f5 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:14.503298 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f57666-528a-488d-826a-cc58282010f5 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:14.508747 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6dde2-bc15-45cc-a1ba-424e1d0144ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:14.509692 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6dde2-bc15-45cc-a1ba-424e1d0144ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:14.509975 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6dde2-bc15-45cc-a1ba-424e1d0144ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:14.510255 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6dde2-bc15-45cc-a1ba-424e1d0144ba admin admin] PATH_INFO: `/v3/projects/2ceeb02ecabb4d5c810819c86e28eb5c/users/4e6b8d07af354ed8866215fb2a217679/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:14.512210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6dde2-bc15-45cc-a1ba-424e1d0144ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ceeb02ecabb4d5c810819c86e28eb5c, user_id=4e6b8d07af354ed8866215fb2a217679, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:14.515160 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6dde2-bc15-45cc-a1ba-424e1d0144ba admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:14.525941 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1279/2540] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:14 2022] PUT /identity/v3/projects/2ceeb02ecabb4d5c810819c86e28eb5c/users/4e6b8d07af354ed8866215fb2a217679/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:14.537332 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b068827-1aed-4798-a51a-ddda8c1e990c None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:14.537820 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b068827-1aed-4798-a51a-ddda8c1e990c None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:14.537918 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b068827-1aed-4798-a51a-ddda8c1e990c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:14.564265 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-6b068827-1aed-4798-a51a-ddda8c1e990c None None] MFA Rules not processed for user `4e6b8d07af354ed8866215fb2a217679`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:14.565420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-6b068827-1aed-4798-a51a-ddda8c1e990c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:14.609630 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:14.633177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1262/2541] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:14.643936 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6dde2-bc15-45cc-a1ba-424e1d0144ba admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:14.645784 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-21c6dde2-bc15-45cc-a1ba-424e1d0144ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:14.708896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6dde2-bc15-45cc-a1ba-424e1d0144ba admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:14.713051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b60c3-3b60-433d-b712-417e9723ac38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:14.713728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b60c3-3b60-433d-b712-417e9723ac38 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:14.714015 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b60c3-3b60-433d-b712-417e9723ac38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:14.714295 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b60c3-3b60-433d-b712-417e9723ac38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:14.723854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b60c3-3b60-433d-b712-417e9723ac38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:14.725331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b60c3-3b60-433d-b712-417e9723ac38 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:14.735373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1280/2542] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:57:15.703156 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b068827-1aed-4798-a51a-ddda8c1e990c None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:15.704540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-6b068827-1aed-4798-a51a-ddda8c1e990c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:15.717457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b068827-1aed-4798-a51a-ddda8c1e990c None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:15.720984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fb680f-ae0f-4181-b6c6-f9197f60f299 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:15.721629 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb680f-ae0f-4181-b6c6-f9197f60f299 service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:15.721914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb680f-ae0f-4181-b6c6-f9197f60f299 service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:15.722203 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fb680f-ae0f-4181-b6c6-f9197f60f299 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:15.777772 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb680f-ae0f-4181-b6c6-f9197f60f299 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:15.779136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fb680f-ae0f-4181-b6c6-f9197f60f299 service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:15.789408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1263/2543] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:57:17.289434 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b60c3-3b60-433d-b712-417e9723ac38 service cinder] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:17.289434 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-0b2b60c3-3b60-433d-b712-417e9723ac38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:17.309089 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:17.353877 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b60c3-3b60-433d-b712-417e9723ac38 service cinder] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:17.358526 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800499e-5bc5-4609-a17e-735999f10b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:17.359404 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800499e-5bc5-4609-a17e-735999f10b0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:17.360045 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800499e-5bc5-4609-a17e-735999f10b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:17.360045 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800499e-5bc5-4609-a17e-735999f10b0d admin admin] PATH_INFO: `/v3/users/d8c613270c654a21aba84643f4844756` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:17.361181 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800499e-5bc5-4609-a17e-735999f10b0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8c613270c654a21aba84643f4844756)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:17.363153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800499e-5bc5-4609-a17e-735999f10b0d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:17.619008 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-2800499e-5bc5-4609-a17e-735999f10b0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8c613270c654a21aba84643f4844756'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:17.631419 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-2800499e-5bc5-4609-a17e-735999f10b0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8c613270c654a21aba84643f4844756'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:17.735472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-2800499e-5bc5-4609-a17e-735999f10b0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8c613270c654a21aba84643f4844756'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:17.736095 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-2800499e-5bc5-4609-a17e-735999f10b0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8c613270c654a21aba84643f4844756'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:17.744141 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-2800499e-5bc5-4609-a17e-735999f10b0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8c613270c654a21aba84643f4844756'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:17.750042 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1281/2544] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:57:17 2022] DELETE /identity/v3/users/d8c613270c654a21aba84643f4844756 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:17.755126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39004a87-f547-4d36-9193-1a995024bac7 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:17.755649 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39004a87-f547-4d36-9193-1a995024bac7 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:17.755922 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39004a87-f547-4d36-9193-1a995024bac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:17.775139 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-39004a87-f547-4d36-9193-1a995024bac7 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:17.776348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-39004a87-f547-4d36-9193-1a995024bac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:17.840205 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1264/2545] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:17.854559 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800499e-5bc5-4609-a17e-735999f10b0d admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:17.855973 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-2800499e-5bc5-4609-a17e-735999f10b0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:17.914630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800499e-5bc5-4609-a17e-735999f10b0d admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:17.919634 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595bbb1-4d22-424f-8c14-0b37828e46c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:17.920286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595bbb1-4d22-424f-8c14-0b37828e46c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:17.920572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595bbb1-4d22-424f-8c14-0b37828e46c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:17.920870 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595bbb1-4d22-424f-8c14-0b37828e46c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:17.931235 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595bbb1-4d22-424f-8c14-0b37828e46c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:17.932682 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a595bbb1-4d22-424f-8c14-0b37828e46c7 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:17.947760 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1282/2546] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:57:18.411207 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39004a87-f547-4d36-9193-1a995024bac7 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:18.412689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-39004a87-f547-4d36-9193-1a995024bac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:18.427053 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39004a87-f547-4d36-9193-1a995024bac7 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:18.432000 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ec84e-d796-4be1-a19e-ada394277558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:18.433154 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ec84e-d796-4be1-a19e-ada394277558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:18.433439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ec84e-d796-4be1-a19e-ada394277558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:18.433716 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ec84e-d796-4be1-a19e-ada394277558 admin admin] PATH_INFO: `/v3/projects/517b8862a76443f09cbee095c54cd4ea` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:18.434908 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ec84e-d796-4be1-a19e-ada394277558 admin admin] RBAC: Authorizing `identity:delete_project(project_id=517b8862a76443f09cbee095c54cd4ea)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:18.436899 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ec84e-d796-4be1-a19e-ada394277558 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:18.541849 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-825ec84e-d796-4be1-a19e-ada394277558 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '517b8862a76443f09cbee095c54cd4ea', 'request_id': 'req-825ec84e-d796-4be1-a19e-ada394277558'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:18.547273 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-825ec84e-d796-4be1-a19e-ada394277558 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '517b8862a76443f09cbee095c54cd4ea', 'request_id': 'req-825ec84e-d796-4be1-a19e-ada394277558'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:18.620788 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-825ec84e-d796-4be1-a19e-ada394277558 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '517b8862a76443f09cbee095c54cd4ea', 'request_id': 'req-825ec84e-d796-4be1-a19e-ada394277558'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:18.632729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-825ec84e-d796-4be1-a19e-ada394277558 admin admin] The token cache is being invalidate because project 517b8862a76443f09cbee095c54cd4ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:57:18.633052 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-825ec84e-d796-4be1-a19e-ada394277558 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:18.633696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-825ec84e-d796-4be1-a19e-ada394277558 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:18.635235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1265/2547] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:57:18 2022] DELETE /identity/v3/projects/517b8862a76443f09cbee095c54cd4ea => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:18.640243 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f5153-a109-4d6f-85e4-f008ce60b1a7 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:18.640736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f5153-a109-4d6f-85e4-f008ce60b1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:18.640844 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270f5153-a109-4d6f-85e4-f008ce60b1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:18.659376 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-270f5153-a109-4d6f-85e4-f008ce60b1a7 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:18.660764 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-270f5153-a109-4d6f-85e4-f008ce60b1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:18.693278 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:18.720014 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1283/2548] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:18.729261 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72db3c-bbe1-4fc0-abca-73f6094dc9da None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:18.729559 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72db3c-bbe1-4fc0-abca-73f6094dc9da None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:18.729846 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72db3c-bbe1-4fc0-abca-73f6094dc9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:18.737451 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:18.749027 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-2c72db3c-bbe1-4fc0-abca-73f6094dc9da None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:18.750268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-2c72db3c-bbe1-4fc0-abca-73f6094dc9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:18.759154 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1266/2549] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:18.763382 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f5153-a109-4d6f-85e4-f008ce60b1a7 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:18.773905 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270f5153-a109-4d6f-85e4-f008ce60b1a7 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:18.778470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7080376-8f64-41f1-a771-0ca13e0b7cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:18.779122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7080376-8f64-41f1-a771-0ca13e0b7cf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:18.779410 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7080376-8f64-41f1-a771-0ca13e0b7cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:18.779729 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7080376-8f64-41f1-a771-0ca13e0b7cf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:18.780479 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7080376-8f64-41f1-a771-0ca13e0b7cf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:57:18.780786 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7080376-8f64-41f1-a771-0ca13e0b7cf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:18.782448 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7080376-8f64-41f1-a771-0ca13e0b7cf3 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:18.794439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1284/2550] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:57:18.798771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72db3c-bbe1-4fc0-abca-73f6094dc9da None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:18.808472 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72db3c-bbe1-4fc0-abca-73f6094dc9da None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:18.813198 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df8eff-a4c4-4b6b-afc7-36a37eb3ce7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:18.813897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df8eff-a4c4-4b6b-afc7-36a37eb3ce7a admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:18.814186 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df8eff-a4c4-4b6b-afc7-36a37eb3ce7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:18.814469 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df8eff-a4c4-4b6b-afc7-36a37eb3ce7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:18.815156 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df8eff-a4c4-4b6b-afc7-36a37eb3ce7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:18.817936 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df8eff-a4c4-4b6b-afc7-36a37eb3ce7a admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:18.952191 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1267/2551] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:57:18 2022] POST /identity/v3/projects => generated 389 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:18.953623 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7080376-8f64-41f1-a771-0ca13e0b7cf3 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:18.964370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7080376-8f64-41f1-a771-0ca13e0b7cf3 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:19.021237 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfa57e-ebc5-45d4-ad83-9e639fc7524e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:19.021906 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfa57e-ebc5-45d4-ad83-9e639fc7524e admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:19.022189 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfa57e-ebc5-45d4-ad83-9e639fc7524e admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:19.022466 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bfa57e-ebc5-45d4-ad83-9e639fc7524e admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:19.023146 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bfa57e-ebc5-45d4-ad83-9e639fc7524e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:19.025093 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bfa57e-ebc5-45d4-ad83-9e639fc7524e admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:19.130059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1285/2552] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:57:18 2022] POST /identity/v3/users => generated 295 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:19.134227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df8eff-a4c4-4b6b-afc7-36a37eb3ce7a admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:19.144407 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df8eff-a4c4-4b6b-afc7-36a37eb3ce7a admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:19.149535 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6e8e2-7f5c-4f1d-83bc-438be3db617f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:19.150191 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6e8e2-7f5c-4f1d-83bc-438be3db617f admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:19.150480 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6e8e2-7f5c-4f1d-83bc-438be3db617f admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:19.150763 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6e8e2-7f5c-4f1d-83bc-438be3db617f admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:19.151488 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6e8e2-7f5c-4f1d-83bc-438be3db617f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:19.153129 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca6e8e2-7f5c-4f1d-83bc-438be3db617f admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:19.164439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1268/2553] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:19.168542 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfa57e-ebc5-45d4-ad83-9e639fc7524e admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:19.178841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bfa57e-ebc5-45d4-ad83-9e639fc7524e admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:19.183554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27ff36-f2f5-4ec4-9cb2-2288ef23a594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:19.184215 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27ff36-f2f5-4ec4-9cb2-2288ef23a594 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:19.184504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27ff36-f2f5-4ec4-9cb2-2288ef23a594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:19.184785 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27ff36-f2f5-4ec4-9cb2-2288ef23a594 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:19.185678 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27ff36-f2f5-4ec4-9cb2-2288ef23a594 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:19.189876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27ff36-f2f5-4ec4-9cb2-2288ef23a594 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:19.197305 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1286/2554] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:19.201541 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6e8e2-7f5c-4f1d-83bc-438be3db617f admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:19.212387 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca6e8e2-7f5c-4f1d-83bc-438be3db617f admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:19.217062 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180cd7d-586d-4230-9a45-4d3ace10a498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:19.217953 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180cd7d-586d-4230-9a45-4d3ace10a498 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:19.221012 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180cd7d-586d-4230-9a45-4d3ace10a498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:19.221012 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180cd7d-586d-4230-9a45-4d3ace10a498 admin admin] PATH_INFO: `/v3/projects/49783f4dd99f45f19a5c418d4dae24ef/users/d8e2817dcf7e43d68721b5434cbaeec5/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:19.234742 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7180cd7d-586d-4230-9a45-4d3ace10a498 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49783f4dd99f45f19a5c418d4dae24ef, user_id=d8e2817dcf7e43d68721b5434cbaeec5, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:19.237576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7180cd7d-586d-4230-9a45-4d3ace10a498 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:19.358419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1269/2555] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:19 2022] PUT /identity/v3/projects/49783f4dd99f45f19a5c418d4dae24ef/users/d8e2817dcf7e43d68721b5434cbaeec5/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:57:19.369400 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e619b4b-24d8-4a5e-9b0e-f590147171d9 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:19.370381 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e619b4b-24d8-4a5e-9b0e-f590147171d9 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:19.370667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e619b4b-24d8-4a5e-9b0e-f590147171d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:19.401371 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-7e619b4b-24d8-4a5e-9b0e-f590147171d9 None None] MFA Rules not processed for user `d8e2817dcf7e43d68721b5434cbaeec5`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:19.402112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-7e619b4b-24d8-4a5e-9b0e-f590147171d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:19.469202 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1287/2556] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:57:19.473970 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180cd7d-586d-4230-9a45-4d3ace10a498 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:19.543223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7180cd7d-586d-4230-9a45-4d3ace10a498 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:19.547869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8185e-53b4-49d3-889e-215ea5d231c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:19.548530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8185e-53b4-49d3-889e-215ea5d231c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:19.548818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8185e-53b4-49d3-889e-215ea5d231c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:19.549138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8185e-53b4-49d3-889e-215ea5d231c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:19.549834 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c8185e-53b4-49d3-889e-215ea5d231c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:19.551810 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c8185e-53b4-49d3-889e-215ea5d231c9 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:19.668622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1270/2557] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:57:19 2022] POST /identity/v3/projects => generated 389 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:19.672625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e619b4b-24d8-4a5e-9b0e-f590147171d9 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:19.683823 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e619b4b-24d8-4a5e-9b0e-f590147171d9 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:19.713009 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:19.739505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01adec49-6a14-49e1-bb86-13839ecd502a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:19.740166 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adec49-6a14-49e1-bb86-13839ecd502a admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:19.740454 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adec49-6a14-49e1-bb86-13839ecd502a admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:19.740733 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01adec49-6a14-49e1-bb86-13839ecd502a admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:19.741464 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01adec49-6a14-49e1-bb86-13839ecd502a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:19.743407 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01adec49-6a14-49e1-bb86-13839ecd502a admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:19.884582 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1288/2558] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:57:19 2022] POST /identity/v3/users => generated 301 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:19.888509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8185e-53b4-49d3-889e-215ea5d231c9 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:19.899206 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8185e-53b4-49d3-889e-215ea5d231c9 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:19.903702 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ef8ff-016c-492e-93d9-6ef9415c7929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:19.904365 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ef8ff-016c-492e-93d9-6ef9415c7929 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:19.904649 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ef8ff-016c-492e-93d9-6ef9415c7929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:19.904929 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ef8ff-016c-492e-93d9-6ef9415c7929 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:19.905692 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ef8ff-016c-492e-93d9-6ef9415c7929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:19.907296 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ef8ff-016c-492e-93d9-6ef9415c7929 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:19.918971 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1271/2559] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:19.923380 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01adec49-6a14-49e1-bb86-13839ecd502a admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:19.933804 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01adec49-6a14-49e1-bb86-13839ecd502a admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:19.938855 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eebb74-2804-47b4-a8a1-fc18ba5d9b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:19.939739 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eebb74-2804-47b4-a8a1-fc18ba5d9b26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:19.940030 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eebb74-2804-47b4-a8a1-fc18ba5d9b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:19.940314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eebb74-2804-47b4-a8a1-fc18ba5d9b26 admin admin] PATH_INFO: `/v3/projects/64c092d7097a4e129c7f99e31ddc24aa/users/28433076be8c4fb3831088c8ac41a416/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:19.955804 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eebb74-2804-47b4-a8a1-fc18ba5d9b26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c092d7097a4e129c7f99e31ddc24aa, user_id=28433076be8c4fb3831088c8ac41a416, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:19.959433 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eebb74-2804-47b4-a8a1-fc18ba5d9b26 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:20.038929 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1289/2560] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:19 2022] PUT /identity/v3/projects/64c092d7097a4e129c7f99e31ddc24aa/users/28433076be8c4fb3831088c8ac41a416/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:20.042908 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ef8ff-016c-492e-93d9-6ef9415c7929 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:20.053070 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ef8ff-016c-492e-93d9-6ef9415c7929 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:20.108308 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2568a313-64ee-411a-a6e4-1f0981b1ce7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:20.108992 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568a313-64ee-411a-a6e4-1f0981b1ce7d admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:20.109279 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568a313-64ee-411a-a6e4-1f0981b1ce7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:20.109557 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2568a313-64ee-411a-a6e4-1f0981b1ce7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:20.110275 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2568a313-64ee-411a-a6e4-1f0981b1ce7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:20.111867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2568a313-64ee-411a-a6e4-1f0981b1ce7d admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:20.121721 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1272/2561] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:20 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:20.126048 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eebb74-2804-47b4-a8a1-fc18ba5d9b26 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:20.137594 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eebb74-2804-47b4-a8a1-fc18ba5d9b26 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:20.141366 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc304b-56c8-4167-a35a-28fc92591040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:20.142243 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc304b-56c8-4167-a35a-28fc92591040 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:20.142548 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc304b-56c8-4167-a35a-28fc92591040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:20.142826 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc304b-56c8-4167-a35a-28fc92591040 admin admin] PATH_INFO: `/v3/projects/64c092d7097a4e129c7f99e31ddc24aa/users/28433076be8c4fb3831088c8ac41a416/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:20.145622 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc304b-56c8-4167-a35a-28fc92591040 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c092d7097a4e129c7f99e31ddc24aa, user_id=28433076be8c4fb3831088c8ac41a416, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:20.148288 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc304b-56c8-4167-a35a-28fc92591040 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:20.158658 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1290/2562] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:20 2022] PUT /identity/v3/projects/64c092d7097a4e129c7f99e31ddc24aa/users/28433076be8c4fb3831088c8ac41a416/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:20.168778 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717c3a1-dcd8-4038-a322-a22ca18a6d4d None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:20.169113 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717c3a1-dcd8-4038-a322-a22ca18a6d4d None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:20.169398 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717c3a1-dcd8-4038-a322-a22ca18a6d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:20.188002 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:20.195662 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-1717c3a1-dcd8-4038-a322-a22ca18a6d4d None None] MFA Rules not processed for user `28433076be8c4fb3831088c8ac41a416`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:20.196857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-1717c3a1-dcd8-4038-a322-a22ca18a6d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:20.266809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1273/2563] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:57:20.278305 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc304b-56c8-4167-a35a-28fc92591040 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:20.279962 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-fefc304b-56c8-4167-a35a-28fc92591040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:20.337892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc304b-56c8-4167-a35a-28fc92591040 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:20.341493 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4ab96-aa1f-45a2-91b8-1f086743b36c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:20.342139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4ab96-aa1f-45a2-91b8-1f086743b36c service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:20.342427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4ab96-aa1f-45a2-91b8-1f086743b36c service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:20.342710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c4ab96-aa1f-45a2-91b8-1f086743b36c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:20.351545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c4ab96-aa1f-45a2-91b8-1f086743b36c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:20.353011 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c4ab96-aa1f-45a2-91b8-1f086743b36c service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:20.362248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1291/2564] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:20 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:57:20.495074 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717c3a1-dcd8-4038-a322-a22ca18a6d4d None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:20.584607 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717c3a1-dcd8-4038-a322-a22ca18a6d4d None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:20.593335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e55e77-5f9d-4f54-87d1-d790ceed10ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:20.594799 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e55e77-5f9d-4f54-87d1-d790ceed10ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:20.595160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e55e77-5f9d-4f54-87d1-d790ceed10ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:20.595690 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e55e77-5f9d-4f54-87d1-d790ceed10ea admin admin] PATH_INFO: `/v3/users/d8e2817dcf7e43d68721b5434cbaeec5` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:20.598056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e55e77-5f9d-4f54-87d1-d790ceed10ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8e2817dcf7e43d68721b5434cbaeec5)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:20.601265 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e55e77-5f9d-4f54-87d1-d790ceed10ea admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:20.838955 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d3e55e77-5f9d-4f54-87d1-d790ceed10ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8e2817dcf7e43d68721b5434cbaeec5'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:20.846682 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4ab96-aa1f-45a2-91b8-1f086743b36c service cinder] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:20.848886 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-02c4ab96-aa1f-45a2-91b8-1f086743b36c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:20.849596 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d3e55e77-5f9d-4f54-87d1-d790ceed10ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8e2817dcf7e43d68721b5434cbaeec5'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:20.859977 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c4ab96-aa1f-45a2-91b8-1f086743b36c service cinder] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:20.863776 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6197fb-7e7a-43c5-bd8d-6cc841d617f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:20.864450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6197fb-7e7a-43c5-bd8d-6cc841d617f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:20.864729 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6197fb-7e7a-43c5-bd8d-6cc841d617f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:20.865071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6197fb-7e7a-43c5-bd8d-6cc841d617f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:20.866803 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-8f6197fb-7e7a-43c5-bd8d-6cc841d617f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:20.924882 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6197fb-7e7a-43c5-bd8d-6cc841d617f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:20.926363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6197fb-7e7a-43c5-bd8d-6cc841d617f4 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:20.935437 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1292/2565] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:57:20.952413 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d3e55e77-5f9d-4f54-87d1-d790ceed10ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8e2817dcf7e43d68721b5434cbaeec5'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:20.958367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d3e55e77-5f9d-4f54-87d1-d790ceed10ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8e2817dcf7e43d68721b5434cbaeec5'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:20.958367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d3e55e77-5f9d-4f54-87d1-d790ceed10ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8e2817dcf7e43d68721b5434cbaeec5'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:20.968350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1274/2566] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:57:20 2022] DELETE /identity/v3/users/d8e2817dcf7e43d68721b5434cbaeec5 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:20.972803 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8ce57-1dc8-49b4-a6b0-27993684bc21 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:20.973343 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8ce57-1dc8-49b4-a6b0-27993684bc21 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:20.973985 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8ce57-1dc8-49b4-a6b0-27993684bc21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:20.991619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-08a8ce57-1dc8-49b4-a6b0-27993684bc21 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:20.992809 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-08a8ce57-1dc8-49b4-a6b0-27993684bc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:21.015035 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:21.058736 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1293/2567] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:21.069062 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e55e77-5f9d-4f54-87d1-d790ceed10ea admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:21.070444 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d3e55e77-5f9d-4f54-87d1-d790ceed10ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:21.089961 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:21.127812 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e55e77-5f9d-4f54-87d1-d790ceed10ea admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:21.131940 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb0c57-de77-478e-a2b6-6fc78fd13704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:21.132590 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb0c57-de77-478e-a2b6-6fc78fd13704 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:21.132873 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb0c57-de77-478e-a2b6-6fc78fd13704 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:21.133185 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb0c57-de77-478e-a2b6-6fc78fd13704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:21.142465 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb0c57-de77-478e-a2b6-6fc78fd13704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:21.143901 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb0c57-de77-478e-a2b6-6fc78fd13704 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:21.154210 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1275/2568] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:57:21.555286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8ce57-1dc8-49b4-a6b0-27993684bc21 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:21.556788 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-08a8ce57-1dc8-49b4-a6b0-27993684bc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:21.571600 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8ce57-1dc8-49b4-a6b0-27993684bc21 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:21.576994 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f4230-86d9-464e-b7d0-e256d6f2f977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:21.579133 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f4230-86d9-464e-b7d0-e256d6f2f977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:21.579133 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f4230-86d9-464e-b7d0-e256d6f2f977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:21.579133 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f4230-86d9-464e-b7d0-e256d6f2f977 admin admin] PATH_INFO: `/v3/projects/49783f4dd99f45f19a5c418d4dae24ef` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:21.580131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f4230-86d9-464e-b7d0-e256d6f2f977 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49783f4dd99f45f19a5c418d4dae24ef)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:21.581518 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f4230-86d9-464e-b7d0-e256d6f2f977 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:21.703272 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-448f4230-86d9-464e-b7d0-e256d6f2f977 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49783f4dd99f45f19a5c418d4dae24ef', 'request_id': 'req-448f4230-86d9-464e-b7d0-e256d6f2f977'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:21.710121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-448f4230-86d9-464e-b7d0-e256d6f2f977 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49783f4dd99f45f19a5c418d4dae24ef', 'request_id': 'req-448f4230-86d9-464e-b7d0-e256d6f2f977'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:21.716940 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-448f4230-86d9-464e-b7d0-e256d6f2f977 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49783f4dd99f45f19a5c418d4dae24ef', 'request_id': 'req-448f4230-86d9-464e-b7d0-e256d6f2f977'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:21.792021 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-448f4230-86d9-464e-b7d0-e256d6f2f977 admin admin] The token cache is being invalidate because project 49783f4dd99f45f19a5c418d4dae24ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:57:21.797469 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-448f4230-86d9-464e-b7d0-e256d6f2f977 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:21.797469 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-448f4230-86d9-464e-b7d0-e256d6f2f977 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:21.797469 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1294/2569] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:57:21 2022] DELETE /identity/v3/projects/49783f4dd99f45f19a5c418d4dae24ef => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:21.801809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb0c57-de77-478e-a2b6-6fc78fd13704 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:21.811522 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb0c57-de77-478e-a2b6-6fc78fd13704 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:21.869255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecc490c-4dbd-4293-b984-09bc2e62f6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:21.870129 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc490c-4dbd-4293-b984-09bc2e62f6a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:21.870524 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc490c-4dbd-4293-b984-09bc2e62f6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:21.870692 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecc490c-4dbd-4293-b984-09bc2e62f6a9 admin admin] PATH_INFO: `/v3/users/28433076be8c4fb3831088c8ac41a416` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:21.871880 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecc490c-4dbd-4293-b984-09bc2e62f6a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28433076be8c4fb3831088c8ac41a416)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:21.873852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecc490c-4dbd-4293-b984-09bc2e62f6a9 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:22.097389 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-9ecc490c-4dbd-4293-b984-09bc2e62f6a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28433076be8c4fb3831088c8ac41a416'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:22.109249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-9ecc490c-4dbd-4293-b984-09bc2e62f6a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28433076be8c4fb3831088c8ac41a416'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:22.209324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-9ecc490c-4dbd-4293-b984-09bc2e62f6a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28433076be8c4fb3831088c8ac41a416'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:22.210029 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-9ecc490c-4dbd-4293-b984-09bc2e62f6a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28433076be8c4fb3831088c8ac41a416'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:22.211124 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-9ecc490c-4dbd-4293-b984-09bc2e62f6a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28433076be8c4fb3831088c8ac41a416'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:22.225230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1276/2570] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:57:21 2022] DELETE /identity/v3/users/28433076be8c4fb3831088c8ac41a416 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:22.633496 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f4230-86d9-464e-b7d0-e256d6f2f977 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:22.636287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-448f4230-86d9-464e-b7d0-e256d6f2f977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:22.650318 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f4230-86d9-464e-b7d0-e256d6f2f977 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:22.707379 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60db81c-fac0-4035-84cd-8f883f81d330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:22.708203 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60db81c-fac0-4035-84cd-8f883f81d330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:22.709122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60db81c-fac0-4035-84cd-8f883f81d330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:22.709561 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60db81c-fac0-4035-84cd-8f883f81d330 admin admin] PATH_INFO: `/v3/projects/64c092d7097a4e129c7f99e31ddc24aa` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:22.711010 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60db81c-fac0-4035-84cd-8f883f81d330 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64c092d7097a4e129c7f99e31ddc24aa)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:22.712933 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60db81c-fac0-4035-84cd-8f883f81d330 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:22.845721 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c60db81c-fac0-4035-84cd-8f883f81d330 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64c092d7097a4e129c7f99e31ddc24aa', 'request_id': 'req-c60db81c-fac0-4035-84cd-8f883f81d330'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:22.847665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:22.852139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c60db81c-fac0-4035-84cd-8f883f81d330 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64c092d7097a4e129c7f99e31ddc24aa', 'request_id': 'req-c60db81c-fac0-4035-84cd-8f883f81d330'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:22.858211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c60db81c-fac0-4035-84cd-8f883f81d330 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64c092d7097a4e129c7f99e31ddc24aa', 'request_id': 'req-c60db81c-fac0-4035-84cd-8f883f81d330'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:22.924658 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c60db81c-fac0-4035-84cd-8f883f81d330 admin admin] The token cache is being invalidate because project 64c092d7097a4e129c7f99e31ddc24aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:57:22.925596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c60db81c-fac0-4035-84cd-8f883f81d330 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:22.925596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-c60db81c-fac0-4035-84cd-8f883f81d330 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:22.927412 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1295/2571] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:57:22 2022] DELETE /identity/v3/projects/64c092d7097a4e129c7f99e31ddc24aa => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:22.932515 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabe9ec-03a0-4946-a98c-b76f1a487f48 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:22.932884 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabe9ec-03a0-4946-a98c-b76f1a487f48 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:22.933306 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cabe9ec-03a0-4946-a98c-b76f1a487f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:22.959681 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-9cabe9ec-03a0-4946-a98c-b76f1a487f48 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:22.961335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-9cabe9ec-03a0-4946-a98c-b76f1a487f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:22.997906 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:23.024551 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1277/2572] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:57:23.034089 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fef66-a6d2-4f95-91e8-548a30ce48c9 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:23.034381 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fef66-a6d2-4f95-91e8-548a30ce48c9 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:23.034666 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1fef66-a6d2-4f95-91e8-548a30ce48c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:23.053658 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-8d1fef66-a6d2-4f95-91e8-548a30ce48c9 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:23.054861 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-8d1fef66-a6d2-4f95-91e8-548a30ce48c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:23.062835 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1296/2573] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:57:23.067107 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabe9ec-03a0-4946-a98c-b76f1a487f48 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:23.077567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cabe9ec-03a0-4946-a98c-b76f1a487f48 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:23.082117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10994c-6f82-4f6d-b8c4-a2f8ec8141bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:23.082787 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10994c-6f82-4f6d-b8c4-a2f8ec8141bf admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:23.083055 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10994c-6f82-4f6d-b8c4-a2f8ec8141bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:23.083493 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10994c-6f82-4f6d-b8c4-a2f8ec8141bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:23.084228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10994c-6f82-4f6d-b8c4-a2f8ec8141bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:57:23.084539 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10994c-6f82-4f6d-b8c4-a2f8ec8141bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:23.086412 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10994c-6f82-4f6d-b8c4-a2f8ec8141bf admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:23.099895 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1278/2574] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:57:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:57:23.105512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fef66-a6d2-4f95-91e8-548a30ce48c9 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:23.113916 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1fef66-a6d2-4f95-91e8-548a30ce48c9 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:23.118908 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebde138-5280-4d8f-b15c-efa0a2a571da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:23.119562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebde138-5280-4d8f-b15c-efa0a2a571da admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:23.119853 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebde138-5280-4d8f-b15c-efa0a2a571da admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:23.120131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebde138-5280-4d8f-b15c-efa0a2a571da admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:23.120828 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebde138-5280-4d8f-b15c-efa0a2a571da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:23.122852 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebde138-5280-4d8f-b15c-efa0a2a571da admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:23.239468 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1297/2575] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:57:23 2022] POST /identity/v3/projects => generated 379 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:23.244035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10994c-6f82-4f6d-b8c4-a2f8ec8141bf admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:23.253800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10994c-6f82-4f6d-b8c4-a2f8ec8141bf admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:23.308880 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d9081-3144-4333-a71e-c31bbb04525b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:23.309528 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d9081-3144-4333-a71e-c31bbb04525b admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:23.309819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d9081-3144-4333-a71e-c31bbb04525b admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:23.310096 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d9081-3144-4333-a71e-c31bbb04525b admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:23.310851 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2d9081-3144-4333-a71e-c31bbb04525b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:23.312813 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2d9081-3144-4333-a71e-c31bbb04525b admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:23.420609 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1279/2576] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:57:23 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:57:23.424825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebde138-5280-4d8f-b15c-efa0a2a571da admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:23.436041 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebde138-5280-4d8f-b15c-efa0a2a571da admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:23.440651 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7655c29-158f-4d3c-b673-e224816c7db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:23.442176 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7655c29-158f-4d3c-b673-e224816c7db5 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:23.442176 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7655c29-158f-4d3c-b673-e224816c7db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:23.442176 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7655c29-158f-4d3c-b673-e224816c7db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:23.442700 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7655c29-158f-4d3c-b673-e224816c7db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:23.444467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7655c29-158f-4d3c-b673-e224816c7db5 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:23.455049 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1298/2577] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:23.459506 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d9081-3144-4333-a71e-c31bbb04525b admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:23.469655 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d9081-3144-4333-a71e-c31bbb04525b admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:23.474915 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff36f9d-de1f-4a01-901e-8bcce19f48f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:23.475559 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff36f9d-de1f-4a01-901e-8bcce19f48f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:23.475842 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff36f9d-de1f-4a01-901e-8bcce19f48f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:23.476116 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff36f9d-de1f-4a01-901e-8bcce19f48f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:23.476833 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff36f9d-de1f-4a01-901e-8bcce19f48f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:23.478600 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff36f9d-de1f-4a01-901e-8bcce19f48f7 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:23.488243 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1280/2578] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:23.492415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7655c29-158f-4d3c-b673-e224816c7db5 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:23.502537 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7655c29-158f-4d3c-b673-e224816c7db5 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:23.507279 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6804494-6c97-4d2f-9cc6-a83f5819de73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:23.508157 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6804494-6c97-4d2f-9cc6-a83f5819de73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:23.508439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6804494-6c97-4d2f-9cc6-a83f5819de73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:23.508715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6804494-6c97-4d2f-9cc6-a83f5819de73 admin admin] PATH_INFO: `/v3/projects/a6358763687c4d309532977109fc7ef1/users/372bdef3dc9744c7839a4f6284fae5ef/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:23.524289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6804494-6c97-4d2f-9cc6-a83f5819de73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6358763687c4d309532977109fc7ef1, user_id=372bdef3dc9744c7839a4f6284fae5ef, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:23.527015 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6804494-6c97-4d2f-9cc6-a83f5819de73 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:23.598137 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1299/2579] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:23 2022] PUT /identity/v3/projects/a6358763687c4d309532977109fc7ef1/users/372bdef3dc9744c7839a4f6284fae5ef/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:23.614059 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dca8ea-e464-4c36-8f1a-e7576f51bde9 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:23.614352 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dca8ea-e464-4c36-8f1a-e7576f51bde9 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:23.614710 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dca8ea-e464-4c36-8f1a-e7576f51bde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:23.641909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-d4dca8ea-e464-4c36-8f1a-e7576f51bde9 None None] MFA Rules not processed for user `372bdef3dc9744c7839a4f6284fae5ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:23.643133 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d4dca8ea-e464-4c36-8f1a-e7576f51bde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:23.711486 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1281/2580] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:23 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:57:23.720026 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6804494-6c97-4d2f-9cc6-a83f5819de73 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:23.781281 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6804494-6c97-4d2f-9cc6-a83f5819de73 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:23.785957 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2c1a5-8276-4ba1-ac61-03bef41139de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:23.787219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2c1a5-8276-4ba1-ac61-03bef41139de admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:23.787512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2c1a5-8276-4ba1-ac61-03bef41139de admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:23.787774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2c1a5-8276-4ba1-ac61-03bef41139de admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:23.788451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2c1a5-8276-4ba1-ac61-03bef41139de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:23.790391 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2c1a5-8276-4ba1-ac61-03bef41139de admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:23.867665 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1300/2581] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:57:23 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:23.871776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dca8ea-e464-4c36-8f1a-e7576f51bde9 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:23.883038 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dca8ea-e464-4c36-8f1a-e7576f51bde9 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:23.913416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:23.939332 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6bc37-5902-49cc-933b-c23b99520294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:23.940042 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6bc37-5902-49cc-933b-c23b99520294 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:23.940330 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6bc37-5902-49cc-933b-c23b99520294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:23.940610 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6bc37-5902-49cc-933b-c23b99520294 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:23.941375 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed6bc37-5902-49cc-933b-c23b99520294 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:23.943907 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed6bc37-5902-49cc-933b-c23b99520294 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:24.015712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1282/2582] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:57:23 2022] POST /identity/v3/users => generated 290 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:57:24.019677 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2c1a5-8276-4ba1-ac61-03bef41139de admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:24.030787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2c1a5-8276-4ba1-ac61-03bef41139de admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:24.035677 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e18105-4292-4dd3-9417-d403d5a890a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:24.036286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e18105-4292-4dd3-9417-d403d5a890a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:24.036570 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e18105-4292-4dd3-9417-d403d5a890a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:24.036847 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e18105-4292-4dd3-9417-d403d5a890a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:24.037612 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e18105-4292-4dd3-9417-d403d5a890a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:24.039231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e18105-4292-4dd3-9417-d403d5a890a0 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:24.050423 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1301/2583] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:24.054836 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6bc37-5902-49cc-933b-c23b99520294 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:24.066383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed6bc37-5902-49cc-933b-c23b99520294 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:24.071866 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542877ef-6bbb-4ead-ba35-288572386eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:24.072570 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542877ef-6bbb-4ead-ba35-288572386eb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:24.072998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542877ef-6bbb-4ead-ba35-288572386eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:24.073287 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542877ef-6bbb-4ead-ba35-288572386eb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:24.074109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542877ef-6bbb-4ead-ba35-288572386eb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:24.075683 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542877ef-6bbb-4ead-ba35-288572386eb9 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:24.097183 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1283/2584] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:24.098375 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e18105-4292-4dd3-9417-d403d5a890a0 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:24.108517 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e18105-4292-4dd3-9417-d403d5a890a0 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:24.113364 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79672a2b-d4d3-48f5-85b8-87e35bd69c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:24.114149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79672a2b-d4d3-48f5-85b8-87e35bd69c7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:24.114467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79672a2b-d4d3-48f5-85b8-87e35bd69c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:24.114751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79672a2b-d4d3-48f5-85b8-87e35bd69c7d admin admin] PATH_INFO: `/v3/projects/d3697365201b41cca975e96251a9e5ee/users/8c41b437a460434891d5c780b4259922/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:24.133092 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79672a2b-d4d3-48f5-85b8-87e35bd69c7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3697365201b41cca975e96251a9e5ee, user_id=8c41b437a460434891d5c780b4259922, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:24.135694 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79672a2b-d4d3-48f5-85b8-87e35bd69c7d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:24.211017 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1302/2585] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:24 2022] PUT /identity/v3/projects/d3697365201b41cca975e96251a9e5ee/users/8c41b437a460434891d5c780b4259922/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:24.221423 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada7c62-8319-4efd-bbed-91049d316a1e None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:24.221924 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada7c62-8319-4efd-bbed-91049d316a1e None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:24.222022 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada7c62-8319-4efd-bbed-91049d316a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:24.249283 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-cada7c62-8319-4efd-bbed-91049d316a1e None None] MFA Rules not processed for user `8c41b437a460434891d5c780b4259922`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:24.250492 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-cada7c62-8319-4efd-bbed-91049d316a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:24.314264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1284/2586] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:24.320891 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79672a2b-d4d3-48f5-85b8-87e35bd69c7d admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:24.350631 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:24.384007 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79672a2b-d4d3-48f5-85b8-87e35bd69c7d admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:24.388789 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337425c-b5eb-4a85-acf2-d3f983e51c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:24.389515 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337425c-b5eb-4a85-acf2-d3f983e51c6e admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:24.389912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337425c-b5eb-4a85-acf2-d3f983e51c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:24.390198 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6337425c-b5eb-4a85-acf2-d3f983e51c6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:24.390970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337425c-b5eb-4a85-acf2-d3f983e51c6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:24.392896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6337425c-b5eb-4a85-acf2-d3f983e51c6e admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:24.468009 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1303/2587] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:57:24 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:24.472103 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada7c62-8319-4efd-bbed-91049d316a1e None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:24.483451 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada7c62-8319-4efd-bbed-91049d316a1e None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:24.540095 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5208e7f-b139-463b-8e5f-2f0f69bba9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:24.540699 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5208e7f-b139-463b-8e5f-2f0f69bba9e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:24.541018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5208e7f-b139-463b-8e5f-2f0f69bba9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:24.541302 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5208e7f-b139-463b-8e5f-2f0f69bba9e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:24.542015 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5208e7f-b139-463b-8e5f-2f0f69bba9e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:24.543969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5208e7f-b139-463b-8e5f-2f0f69bba9e9 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:24.628818 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337425c-b5eb-4a85-acf2-d3f983e51c6e admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:24.631495 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-6337425c-b5eb-4a85-acf2-d3f983e51c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:24.644052 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6337425c-b5eb-4a85-acf2-d3f983e51c6e admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:24.644871 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1285/2588] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:57:24 2022] POST /identity/v3/users => generated 296 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:57:24.649088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5208e7f-b139-463b-8e5f-2f0f69bba9e9 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:24.649942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6dfa25-9d9e-4b55-bbf4-e817ca138995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:24.650818 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6dfa25-9d9e-4b55-bbf4-e817ca138995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:24.651107 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6dfa25-9d9e-4b55-bbf4-e817ca138995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:24.651389 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6dfa25-9d9e-4b55-bbf4-e817ca138995 admin admin] PATH_INFO: `/v3/users/0b1c961e895b46fa98c6a00a34cad01c` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:24.652609 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6dfa25-9d9e-4b55-bbf4-e817ca138995 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b1c961e895b46fa98c6a00a34cad01c)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:24.654584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6dfa25-9d9e-4b55-bbf4-e817ca138995 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:24.658631 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5208e7f-b139-463b-8e5f-2f0f69bba9e9 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:24.663367 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c7e09-f06a-40c0-ab12-1363dfa33aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:24.664020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c7e09-f06a-40c0-ab12-1363dfa33aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:24.664306 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c7e09-f06a-40c0-ab12-1363dfa33aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:24.664584 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c7e09-f06a-40c0-ab12-1363dfa33aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:24.665345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c7e09-f06a-40c0-ab12-1363dfa33aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:24.666967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c7e09-f06a-40c0-ab12-1363dfa33aa6 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:24.677146 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1286/2589] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:57:24.681167 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c7e09-f06a-40c0-ab12-1363dfa33aa6 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:24.690988 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c7e09-f06a-40c0-ab12-1363dfa33aa6 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:24.696284 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66f569-0043-4e5b-b512-af58cf662d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:24.697180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66f569-0043-4e5b-b512-af58cf662d67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:24.697469 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66f569-0043-4e5b-b512-af58cf662d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:24.697749 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66f569-0043-4e5b-b512-af58cf662d67 admin admin] PATH_INFO: `/v3/projects/fbca9f36903d49cba238b8eb2c171e26/users/9a4caa4530114f14ad208acf74fc73fd/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:24.712999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f66f569-0043-4e5b-b512-af58cf662d67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbca9f36903d49cba238b8eb2c171e26, user_id=9a4caa4530114f14ad208acf74fc73fd, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:24.715630 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f66f569-0043-4e5b-b512-af58cf662d67 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:24.718674 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:24.786460 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1287/2590] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:24 2022] PUT /identity/v3/projects/fbca9f36903d49cba238b8eb2c171e26/users/9a4caa4530114f14ad208acf74fc73fd/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:57:24.790289 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66f569-0043-4e5b-b512-af58cf662d67 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:24.811829 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-cf6dfa25-9d9e-4b55-bbf4-e817ca138995 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b1c961e895b46fa98c6a00a34cad01c'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:24.826120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-cf6dfa25-9d9e-4b55-bbf4-e817ca138995 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b1c961e895b46fa98c6a00a34cad01c'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:24.853058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66f569-0043-4e5b-b512-af58cf662d67 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:24.857938 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7392c5f-7386-4af1-bded-0175e0f34331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:24.858605 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7392c5f-7386-4af1-bded-0175e0f34331 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:24.858892 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7392c5f-7386-4af1-bded-0175e0f34331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:24.859172 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7392c5f-7386-4af1-bded-0175e0f34331 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:24.859930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7392c5f-7386-4af1-bded-0175e0f34331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:24.861581 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7392c5f-7386-4af1-bded-0175e0f34331 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:24.871503 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1288/2591] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:57:24.875480 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7392c5f-7386-4af1-bded-0175e0f34331 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:24.886134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7392c5f-7386-4af1-bded-0175e0f34331 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:24.890835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ff628-a62b-4236-b300-b191e190b868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:24.891696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ff628-a62b-4236-b300-b191e190b868 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:24.891979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ff628-a62b-4236-b300-b191e190b868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:24.892255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ff628-a62b-4236-b300-b191e190b868 admin admin] PATH_INFO: `/v3/projects/fbca9f36903d49cba238b8eb2c171e26/users/9a4caa4530114f14ad208acf74fc73fd/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:24.894255 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ff628-a62b-4236-b300-b191e190b868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbca9f36903d49cba238b8eb2c171e26, user_id=9a4caa4530114f14ad208acf74fc73fd, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:24.897530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ff628-a62b-4236-b300-b191e190b868 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:24.906620 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-cf6dfa25-9d9e-4b55-bbf4-e817ca138995 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b1c961e895b46fa98c6a00a34cad01c'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:24.907293 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-cf6dfa25-9d9e-4b55-bbf4-e817ca138995 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b1c961e895b46fa98c6a00a34cad01c'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:24.907692 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1289/2592] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:24 2022] PUT /identity/v3/projects/fbca9f36903d49cba238b8eb2c171e26/users/9a4caa4530114f14ad208acf74fc73fd/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:57:24.915036 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-cf6dfa25-9d9e-4b55-bbf4-e817ca138995 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b1c961e895b46fa98c6a00a34cad01c'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:24.917905 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952cc3a4-06d8-4438-bf98-8e2cda4a3a33 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:24.918226 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952cc3a4-06d8-4438-bf98-8e2cda4a3a33 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:24.920890 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952cc3a4-06d8-4438-bf98-8e2cda4a3a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:24.921358 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1304/2593] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:57:24 2022] DELETE /identity/v3/users/0b1c961e895b46fa98c6a00a34cad01c => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:57:24.925510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93dbe2f-2ec0-4e41-9399-6f168a7c9c5f None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:24.925802 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93dbe2f-2ec0-4e41-9399-6f168a7c9c5f None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:24.926088 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93dbe2f-2ec0-4e41-9399-6f168a7c9c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:24.945292 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-e93dbe2f-2ec0-4e41-9399-6f168a7c9c5f None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:24.946504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-e93dbe2f-2ec0-4e41-9399-6f168a7c9c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:24.955798 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-952cc3a4-06d8-4438-bf98-8e2cda4a3a33 None None] MFA Rules not processed for user `9a4caa4530114f14ad208acf74fc73fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:24.955798 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-952cc3a4-06d8-4438-bf98-8e2cda4a3a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:25.008009 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1305/2594] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:57:25.017981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93dbe2f-2ec0-4e41-9399-6f168a7c9c5f None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:25.019666 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-e93dbe2f-2ec0-4e41-9399-6f168a7c9c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:25.020648 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1290/2595] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:24 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:25.031754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952cc3a4-06d8-4438-bf98-8e2cda4a3a33 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:25.033164 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-952cc3a4-06d8-4438-bf98-8e2cda4a3a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:25.047113 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:25.079190 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93dbe2f-2ec0-4e41-9399-6f168a7c9c5f None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:25.083833 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9697c-d9ad-4da2-a08e-591137a24064 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:25.084593 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9697c-d9ad-4da2-a08e-591137a24064 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:25.085568 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9697c-d9ad-4da2-a08e-591137a24064 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:25.085854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd9697c-d9ad-4da2-a08e-591137a24064 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:25.092880 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952cc3a4-06d8-4438-bf98-8e2cda4a3a33 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:25.095145 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9697c-d9ad-4da2-a08e-591137a24064 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:25.096583 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9a36b-5247-4e0f-88fb-8b95586296fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:25.096752 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd9697c-d9ad-4da2-a08e-591137a24064 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:25.097260 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9a36b-5247-4e0f-88fb-8b95586296fc service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:25.097546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9a36b-5247-4e0f-88fb-8b95586296fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:25.097827 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9a36b-5247-4e0f-88fb-8b95586296fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:25.107253 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9a36b-5247-4e0f-88fb-8b95586296fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:25.107672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1306/2596] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:57:25.108684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9a36b-5247-4e0f-88fb-8b95586296fc service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:25.119444 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1291/2597] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:57:25.238011 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9697c-d9ad-4da2-a08e-591137a24064 service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:25.238962 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-9fd9697c-d9ad-4da2-a08e-591137a24064 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:25.251221 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd9697c-d9ad-4da2-a08e-591137a24064 service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:25.255131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c5f3f-ed1c-41d5-bfe4-6897f846f7d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:25.260834 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c5f3f-ed1c-41d5-bfe4-6897f846f7d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:25.260834 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c5f3f-ed1c-41d5-bfe4-6897f846f7d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:25.260834 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5c5f3f-ed1c-41d5-bfe4-6897f846f7d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:25.260834 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-bf5c5f3f-ed1c-41d5-bfe4-6897f846f7d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:25.331107 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c5f3f-ed1c-41d5-bfe4-6897f846f7d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:25.332547 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5c5f3f-ed1c-41d5-bfe4-6897f846f7d0 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:25.345066 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1307/2598] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:57:25.527015 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9a36b-5247-4e0f-88fb-8b95586296fc service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:25.529230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-75e9a36b-5247-4e0f-88fb-8b95586296fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:25.547729 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9a36b-5247-4e0f-88fb-8b95586296fc service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:25.552802 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea457d-88e7-4291-a960-67a829ab718f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:25.554420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea457d-88e7-4291-a960-67a829ab718f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:25.554714 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea457d-88e7-4291-a960-67a829ab718f admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:25.555163 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea457d-88e7-4291-a960-67a829ab718f admin admin] PATH_INFO: `/v3/projects/48aa5eceffe7496aa7d662acffbae98b` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:25.556540 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea457d-88e7-4291-a960-67a829ab718f admin admin] RBAC: Authorizing `identity:delete_project(project_id=48aa5eceffe7496aa7d662acffbae98b)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:25.559125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea457d-88e7-4291-a960-67a829ab718f admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:25.560717 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:25.713005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-88ea457d-88e7-4291-a960-67a829ab718f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48aa5eceffe7496aa7d662acffbae98b', 'request_id': 'req-88ea457d-88e7-4291-a960-67a829ab718f'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:25.718249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-88ea457d-88e7-4291-a960-67a829ab718f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48aa5eceffe7496aa7d662acffbae98b', 'request_id': 'req-88ea457d-88e7-4291-a960-67a829ab718f'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:25.780601 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-88ea457d-88e7-4291-a960-67a829ab718f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48aa5eceffe7496aa7d662acffbae98b', 'request_id': 'req-88ea457d-88e7-4291-a960-67a829ab718f'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:25.792390 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-88ea457d-88e7-4291-a960-67a829ab718f admin admin] The token cache is being invalidate because project 48aa5eceffe7496aa7d662acffbae98b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:57:25.792682 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-88ea457d-88e7-4291-a960-67a829ab718f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:25.793323 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-88ea457d-88e7-4291-a960-67a829ab718f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:25.794853 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1292/2599] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:57:25 2022] DELETE /identity/v3/projects/48aa5eceffe7496aa7d662acffbae98b => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:25.798937 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c5f3f-ed1c-41d5-bfe4-6897f846f7d0 service cinder] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:25.810051 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5c5f3f-ed1c-41d5-bfe4-6897f846f7d0 service cinder] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:25.867843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c70510-cd89-48e7-a2c2-98229a8e19e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:25.868718 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c70510-cd89-48e7-a2c2-98229a8e19e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:25.869031 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c70510-cd89-48e7-a2c2-98229a8e19e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:25.869310 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c70510-cd89-48e7-a2c2-98229a8e19e7 admin admin] PATH_INFO: `/v3/users/6143e2076f1e4309893e7878c6171be1` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:25.870477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c70510-cd89-48e7-a2c2-98229a8e19e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6143e2076f1e4309893e7878c6171be1)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:25.872472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c70510-cd89-48e7-a2c2-98229a8e19e7 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:26.079925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-73c70510-cd89-48e7-a2c2-98229a8e19e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6143e2076f1e4309893e7878c6171be1'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:26.092376 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-73c70510-cd89-48e7-a2c2-98229a8e19e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6143e2076f1e4309893e7878c6171be1'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:26.095695 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:26.168658 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-73c70510-cd89-48e7-a2c2-98229a8e19e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6143e2076f1e4309893e7878c6171be1'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:26.169294 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-73c70510-cd89-48e7-a2c2-98229a8e19e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6143e2076f1e4309893e7878c6171be1'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:26.176641 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-73c70510-cd89-48e7-a2c2-98229a8e19e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6143e2076f1e4309893e7878c6171be1'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:26.183400 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1308/2600] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:57:25 2022] DELETE /identity/v3/users/6143e2076f1e4309893e7878c6171be1 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:26.713847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea457d-88e7-4291-a960-67a829ab718f admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:26.716604 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-88ea457d-88e7-4291-a960-67a829ab718f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:26.729740 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea457d-88e7-4291-a960-67a829ab718f admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:26.786052 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a443d2-a863-4729-ba26-ecea66a68a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:26.787425 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a443d2-a863-4729-ba26-ecea66a68a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:26.787708 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a443d2-a863-4729-ba26-ecea66a68a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:26.787980 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a443d2-a863-4729-ba26-ecea66a68a15 admin admin] PATH_INFO: `/v3/projects/fd7fcd9619d246bcba766a53d4022c64` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:26.789190 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a443d2-a863-4729-ba26-ecea66a68a15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd7fcd9619d246bcba766a53d4022c64)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:26.791087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a443d2-a863-4729-ba26-ecea66a68a15 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:26.876693 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f0a443d2-a863-4729-ba26-ecea66a68a15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd7fcd9619d246bcba766a53d4022c64', 'request_id': 'req-f0a443d2-a863-4729-ba26-ecea66a68a15'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:26.882920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f0a443d2-a863-4729-ba26-ecea66a68a15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd7fcd9619d246bcba766a53d4022c64', 'request_id': 'req-f0a443d2-a863-4729-ba26-ecea66a68a15'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:26.945887 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f0a443d2-a863-4729-ba26-ecea66a68a15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd7fcd9619d246bcba766a53d4022c64', 'request_id': 'req-f0a443d2-a863-4729-ba26-ecea66a68a15'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:26.957267 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f0a443d2-a863-4729-ba26-ecea66a68a15 admin admin] The token cache is being invalidate because project fd7fcd9619d246bcba766a53d4022c64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:57:26.957581 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f0a443d2-a863-4729-ba26-ecea66a68a15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:26.958868 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f0a443d2-a863-4729-ba26-ecea66a68a15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:26.960786 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1293/2601] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:57:26 2022] DELETE /identity/v3/projects/fd7fcd9619d246bcba766a53d4022c64 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:26.964712 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c70510-cd89-48e7-a2c2-98229a8e19e7 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:26.966217 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-73c70510-cd89-48e7-a2c2-98229a8e19e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:26.978899 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c70510-cd89-48e7-a2c2-98229a8e19e7 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:27.034361 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf17fe8-3cf6-4bb7-965c-cc63782391fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:27.035228 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf17fe8-3cf6-4bb7-965c-cc63782391fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:27.035513 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf17fe8-3cf6-4bb7-965c-cc63782391fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:27.035788 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf17fe8-3cf6-4bb7-965c-cc63782391fb admin admin] PATH_INFO: `/v3/users/4e6b8d07af354ed8866215fb2a217679` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:27.037254 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf17fe8-3cf6-4bb7-965c-cc63782391fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e6b8d07af354ed8866215fb2a217679)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:27.039120 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf17fe8-3cf6-4bb7-965c-cc63782391fb admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:27.216668 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-4cf17fe8-3cf6-4bb7-965c-cc63782391fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e6b8d07af354ed8866215fb2a217679'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:27.236146 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-4cf17fe8-3cf6-4bb7-965c-cc63782391fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e6b8d07af354ed8866215fb2a217679'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:27.334835 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-4cf17fe8-3cf6-4bb7-965c-cc63782391fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e6b8d07af354ed8866215fb2a217679'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:27.335723 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-4cf17fe8-3cf6-4bb7-965c-cc63782391fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e6b8d07af354ed8866215fb2a217679'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:27.343516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-4cf17fe8-3cf6-4bb7-965c-cc63782391fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e6b8d07af354ed8866215fb2a217679'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:27.350479 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1309/2602] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:57:26 2022] DELETE /identity/v3/users/4e6b8d07af354ed8866215fb2a217679 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:27.873928 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a443d2-a863-4729-ba26-ecea66a68a15 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:27.876377 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-f0a443d2-a863-4729-ba26-ecea66a68a15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:27.891190 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a443d2-a863-4729-ba26-ecea66a68a15 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:27.919029 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:27.965179 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e576c-ae50-48d4-8377-3f93665f0fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:27.966118 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e576c-ae50-48d4-8377-3f93665f0fc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:27.966413 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e576c-ae50-48d4-8377-3f93665f0fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:27.966693 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e576c-ae50-48d4-8377-3f93665f0fc3 admin admin] PATH_INFO: `/v3/projects/2ceeb02ecabb4d5c810819c86e28eb5c` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:27.967956 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e576c-ae50-48d4-8377-3f93665f0fc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ceeb02ecabb4d5c810819c86e28eb5c)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:27.969973 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055e576c-ae50-48d4-8377-3f93665f0fc3 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:28.078438 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-055e576c-ae50-48d4-8377-3f93665f0fc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ceeb02ecabb4d5c810819c86e28eb5c', 'request_id': 'req-055e576c-ae50-48d4-8377-3f93665f0fc3'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:28.083775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-055e576c-ae50-48d4-8377-3f93665f0fc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ceeb02ecabb4d5c810819c86e28eb5c', 'request_id': 'req-055e576c-ae50-48d4-8377-3f93665f0fc3'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:28.191998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-055e576c-ae50-48d4-8377-3f93665f0fc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ceeb02ecabb4d5c810819c86e28eb5c', 'request_id': 'req-055e576c-ae50-48d4-8377-3f93665f0fc3'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:28.204315 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-055e576c-ae50-48d4-8377-3f93665f0fc3 admin admin] The token cache is being invalidate because project 2ceeb02ecabb4d5c810819c86e28eb5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:57:28.204608 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-055e576c-ae50-48d4-8377-3f93665f0fc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:28.205316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-055e576c-ae50-48d4-8377-3f93665f0fc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:28.206897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1294/2603] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:57:27 2022] DELETE /identity/v3/projects/2ceeb02ecabb4d5c810819c86e28eb5c => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:28.214527 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02906b98-df61-48f5-aedc-bd12664bfc45 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:28.215069 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02906b98-df61-48f5-aedc-bd12664bfc45 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:28.221091 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02906b98-df61-48f5-aedc-bd12664bfc45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:28.233325 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-02906b98-df61-48f5-aedc-bd12664bfc45 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:28.234515 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-02906b98-df61-48f5-aedc-bd12664bfc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:28.298531 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1310/2604] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:28.308061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483c8e6-f7b9-4db6-aa45-f0e5fe675330 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:28.308555 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483c8e6-f7b9-4db6-aa45-f0e5fe675330 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:28.309958 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1483c8e6-f7b9-4db6-aa45-f0e5fe675330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:28.330832 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-1483c8e6-f7b9-4db6-aa45-f0e5fe675330 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:28.332093 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-1483c8e6-f7b9-4db6-aa45-f0e5fe675330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:28.340518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1295/2605] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:57:28.350472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02906b98-df61-48f5-aedc-bd12664bfc45 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:28.356481 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:28.360202 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02906b98-df61-48f5-aedc-bd12664bfc45 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:28.364897 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c52e8b-47c4-48e1-87f5-0d69f2fba976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:28.366131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c52e8b-47c4-48e1-87f5-0d69f2fba976 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:28.366431 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c52e8b-47c4-48e1-87f5-0d69f2fba976 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:28.366714 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c52e8b-47c4-48e1-87f5-0d69f2fba976 admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:28.367447 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c52e8b-47c4-48e1-87f5-0d69f2fba976 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:57:28.367760 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c52e8b-47c4-48e1-87f5-0d69f2fba976 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:28.369471 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c52e8b-47c4-48e1-87f5-0d69f2fba976 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:28.381931 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1311/2606] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:57:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:57:28.386846 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483c8e6-f7b9-4db6-aa45-f0e5fe675330 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:28.396801 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1483c8e6-f7b9-4db6-aa45-f0e5fe675330 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:28.401939 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc73676-c830-49b6-a098-75e68d03c68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:28.409287 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc73676-c830-49b6-a098-75e68d03c68a admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:28.409287 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc73676-c830-49b6-a098-75e68d03c68a admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:28.409287 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc73676-c830-49b6-a098-75e68d03c68a admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:28.409287 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc73676-c830-49b6-a098-75e68d03c68a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:28.409287 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc73676-c830-49b6-a098-75e68d03c68a admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:28.521990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1296/2607] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:57:28 2022] POST /identity/v3/projects => generated 375 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:57:28.525956 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c52e8b-47c4-48e1-87f5-0d69f2fba976 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:28.536394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c52e8b-47c4-48e1-87f5-0d69f2fba976 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:28.594469 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb6069-42ce-48f2-b119-e2d7903ae461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:28.595148 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb6069-42ce-48f2-b119-e2d7903ae461 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:28.595430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb6069-42ce-48f2-b119-e2d7903ae461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:28.595712 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb6069-42ce-48f2-b119-e2d7903ae461 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:28.596595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb6069-42ce-48f2-b119-e2d7903ae461 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:28.598616 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb6069-42ce-48f2-b119-e2d7903ae461 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:28.709474 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1312/2608] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:57:28 2022] POST /identity/v3/users => generated 288 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:28.711426 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc73676-c830-49b6-a098-75e68d03c68a admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:28.722100 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc73676-c830-49b6-a098-75e68d03c68a admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:28.727007 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fca38-c5e4-4cb1-9906-1951549611ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:28.727920 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fca38-c5e4-4cb1-9906-1951549611ee admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:28.728211 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fca38-c5e4-4cb1-9906-1951549611ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:28.728493 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fca38-c5e4-4cb1-9906-1951549611ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:28.729351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759fca38-c5e4-4cb1-9906-1951549611ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:28.731033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759fca38-c5e4-4cb1-9906-1951549611ee admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:28.742123 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1297/2609] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:28.746469 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb6069-42ce-48f2-b119-e2d7903ae461 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:28.757302 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb6069-42ce-48f2-b119-e2d7903ae461 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:28.762210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99832eec-3843-4774-aee8-f49fb38c8165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:28.762873 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99832eec-3843-4774-aee8-f49fb38c8165 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:28.763157 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99832eec-3843-4774-aee8-f49fb38c8165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:28.763429 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99832eec-3843-4774-aee8-f49fb38c8165 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:28.764304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99832eec-3843-4774-aee8-f49fb38c8165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:28.765782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99832eec-3843-4774-aee8-f49fb38c8165 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:28.779655 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1313/2610] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:28.781426 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fca38-c5e4-4cb1-9906-1951549611ee admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:28.791823 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fca38-c5e4-4cb1-9906-1951549611ee admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:28.797068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae5062-9bd6-479e-a3ff-dd1e8d2e0ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:28.798128 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae5062-9bd6-479e-a3ff-dd1e8d2e0ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:28.798542 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae5062-9bd6-479e-a3ff-dd1e8d2e0ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:28.798828 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae5062-9bd6-479e-a3ff-dd1e8d2e0ccb admin admin] PATH_INFO: `/v3/projects/6c7809ae5cde4920a3473efcb60d2f22/users/c97b80f94f424a2ca5622e066276515a/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:28.815252 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae5062-9bd6-479e-a3ff-dd1e8d2e0ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c7809ae5cde4920a3473efcb60d2f22, user_id=c97b80f94f424a2ca5622e066276515a, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:28.818530 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae5062-9bd6-479e-a3ff-dd1e8d2e0ccb admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:28.902063 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1298/2611] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:28 2022] PUT /identity/v3/projects/6c7809ae5cde4920a3473efcb60d2f22/users/c97b80f94f424a2ca5622e066276515a/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:28.914790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b633d-9fbb-45bf-8e25-2c97ce9980a9 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:28.915078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b633d-9fbb-45bf-8e25-2c97ce9980a9 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:28.915360 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b633d-9fbb-45bf-8e25-2c97ce9980a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:28.945335 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-d65b633d-9fbb-45bf-8e25-2c97ce9980a9 None None] MFA Rules not processed for user `c97b80f94f424a2ca5622e066276515a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:28.946604 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d65b633d-9fbb-45bf-8e25-2c97ce9980a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:29.016628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1314/2612] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:29.028586 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae5062-9bd6-479e-a3ff-dd1e8d2e0ccb admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:29.030400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-f8ae5062-9bd6-479e-a3ff-dd1e8d2e0ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:29.082335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:29.088109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae5062-9bd6-479e-a3ff-dd1e8d2e0ccb admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:29.091836 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f303d4d8-71c1-420a-9632-cf3dfb77b517 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:29.101134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303d4d8-71c1-420a-9632-cf3dfb77b517 service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:29.101134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303d4d8-71c1-420a-9632-cf3dfb77b517 service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:29.101134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f303d4d8-71c1-420a-9632-cf3dfb77b517 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:29.101435 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303d4d8-71c1-420a-9632-cf3dfb77b517 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:29.102687 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f303d4d8-71c1-420a-9632-cf3dfb77b517 service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:29.112622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1299/2613] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:57:29.339415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b633d-9fbb-45bf-8e25-2c97ce9980a9 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:29.340883 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d65b633d-9fbb-45bf-8e25-2c97ce9980a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:29.407202 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b633d-9fbb-45bf-8e25-2c97ce9980a9 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:29.411776 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cc7a9-eef0-457c-9770-53ae9402ba33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:29.412664 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cc7a9-eef0-457c-9770-53ae9402ba33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:29.413554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cc7a9-eef0-457c-9770-53ae9402ba33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:29.414151 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cc7a9-eef0-457c-9770-53ae9402ba33 admin admin] PATH_INFO: `/v3/users/372bdef3dc9744c7839a4f6284fae5ef` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:29.415298 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1cc7a9-eef0-457c-9770-53ae9402ba33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=372bdef3dc9744c7839a4f6284fae5ef)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:29.417364 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1cc7a9-eef0-457c-9770-53ae9402ba33 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:29.418944 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:29.622029 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-0f1cc7a9-eef0-457c-9770-53ae9402ba33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '372bdef3dc9744c7839a4f6284fae5ef'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:29.634783 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-0f1cc7a9-eef0-457c-9770-53ae9402ba33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '372bdef3dc9744c7839a4f6284fae5ef'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:29.722793 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-0f1cc7a9-eef0-457c-9770-53ae9402ba33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '372bdef3dc9744c7839a4f6284fae5ef'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:29.723435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-0f1cc7a9-eef0-457c-9770-53ae9402ba33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '372bdef3dc9744c7839a4f6284fae5ef'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:29.730941 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-0f1cc7a9-eef0-457c-9770-53ae9402ba33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '372bdef3dc9744c7839a4f6284fae5ef'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:29.736992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1315/2614] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:57:29 2022] DELETE /identity/v3/users/372bdef3dc9744c7839a4f6284fae5ef => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:57:29.741659 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bcc0d-6b2b-4e39-9786-f209c8ec096b None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:29.741953 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bcc0d-6b2b-4e39-9786-f209c8ec096b None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:29.742235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869bcc0d-6b2b-4e39-9786-f209c8ec096b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:29.763732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-869bcc0d-6b2b-4e39-9786-f209c8ec096b None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:29.764931 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-869bcc0d-6b2b-4e39-9786-f209c8ec096b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:29.826570 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1300/2615] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:57:29.836194 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cc7a9-eef0-457c-9770-53ae9402ba33 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:29.837829 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-0f1cc7a9-eef0-457c-9770-53ae9402ba33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:29.896334 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cc7a9-eef0-457c-9770-53ae9402ba33 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:29.900464 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18ed8f-f44a-4354-8e68-de5a4dcbbcda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:29.901150 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18ed8f-f44a-4354-8e68-de5a4dcbbcda service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:29.901446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18ed8f-f44a-4354-8e68-de5a4dcbbcda service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:29.901724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d18ed8f-f44a-4354-8e68-de5a4dcbbcda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:29.910884 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18ed8f-f44a-4354-8e68-de5a4dcbbcda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:29.912324 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d18ed8f-f44a-4354-8e68-de5a4dcbbcda service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:29.922477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1316/2616] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:57:30.322409 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869bcc0d-6b2b-4e39-9786-f209c8ec096b None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:30.322409 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-869bcc0d-6b2b-4e39-9786-f209c8ec096b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:30.345813 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869bcc0d-6b2b-4e39-9786-f209c8ec096b None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:30.348227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b5d4f-c4a4-4135-bee0-016c67113698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:30.349330 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b5d4f-c4a4-4135-bee0-016c67113698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:30.349642 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b5d4f-c4a4-4135-bee0-016c67113698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:30.349896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b5d4f-c4a4-4135-bee0-016c67113698 admin admin] PATH_INFO: `/v3/projects/a6358763687c4d309532977109fc7ef1` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:30.351817 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530b5d4f-c4a4-4135-bee0-016c67113698 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6358763687c4d309532977109fc7ef1)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:30.353785 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530b5d4f-c4a4-4135-bee0-016c67113698 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:30.467128 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-530b5d4f-c4a4-4135-bee0-016c67113698 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6358763687c4d309532977109fc7ef1', 'request_id': 'req-530b5d4f-c4a4-4135-bee0-016c67113698'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:30.472713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-530b5d4f-c4a4-4135-bee0-016c67113698 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6358763687c4d309532977109fc7ef1', 'request_id': 'req-530b5d4f-c4a4-4135-bee0-016c67113698'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:30.529184 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-530b5d4f-c4a4-4135-bee0-016c67113698 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6358763687c4d309532977109fc7ef1', 'request_id': 'req-530b5d4f-c4a4-4135-bee0-016c67113698'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:30.541863 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-530b5d4f-c4a4-4135-bee0-016c67113698 admin admin] The token cache is being invalidate because project a6358763687c4d309532977109fc7ef1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:57:30.542077 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-530b5d4f-c4a4-4135-bee0-016c67113698 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:30.542713 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-530b5d4f-c4a4-4135-bee0-016c67113698 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:30.544257 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1301/2617] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:57:30 2022] DELETE /identity/v3/projects/a6358763687c4d309532977109fc7ef1 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:57:30.548394 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18ed8f-f44a-4354-8e68-de5a4dcbbcda service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:30.559000 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d18ed8f-f44a-4354-8e68-de5a4dcbbcda service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:30.614637 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be45f0-a6c6-4913-a9f7-181ca1ebff07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:30.615508 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be45f0-a6c6-4913-a9f7-181ca1ebff07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:30.615806 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be45f0-a6c6-4913-a9f7-181ca1ebff07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:30.616080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be45f0-a6c6-4913-a9f7-181ca1ebff07 admin admin] PATH_INFO: `/v3/users/8c41b437a460434891d5c780b4259922` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:30.617290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be45f0-a6c6-4913-a9f7-181ca1ebff07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c41b437a460434891d5c780b4259922)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:30.619217 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be45f0-a6c6-4913-a9f7-181ca1ebff07 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:30.640502 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:30.863076 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-08be45f0-a6c6-4913-a9f7-181ca1ebff07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c41b437a460434891d5c780b4259922'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:30.875572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-08be45f0-a6c6-4913-a9f7-181ca1ebff07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c41b437a460434891d5c780b4259922'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:30.979969 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-08be45f0-a6c6-4913-a9f7-181ca1ebff07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c41b437a460434891d5c780b4259922'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:30.980582 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-08be45f0-a6c6-4913-a9f7-181ca1ebff07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c41b437a460434891d5c780b4259922'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:30.988194 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-08be45f0-a6c6-4913-a9f7-181ca1ebff07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c41b437a460434891d5c780b4259922'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:30.994815 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1317/2618] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:57:30 2022] DELETE /identity/v3/users/8c41b437a460434891d5c780b4259922 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:31.531541 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b5d4f-c4a4-4135-bee0-016c67113698 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:31.533088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-530b5d4f-c4a4-4135-bee0-016c67113698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:31.550180 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b5d4f-c4a4-4135-bee0-016c67113698 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:31.610868 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60093281-146c-4d49-8483-5c1a03e5895f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:31.611660 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60093281-146c-4d49-8483-5c1a03e5895f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:31.611946 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60093281-146c-4d49-8483-5c1a03e5895f admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:31.612223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60093281-146c-4d49-8483-5c1a03e5895f admin admin] PATH_INFO: `/v3/projects/d3697365201b41cca975e96251a9e5ee` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:31.613445 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60093281-146c-4d49-8483-5c1a03e5895f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3697365201b41cca975e96251a9e5ee)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:31.615358 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60093281-146c-4d49-8483-5c1a03e5895f admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:31.632446 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:31.726220 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-60093281-146c-4d49-8483-5c1a03e5895f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3697365201b41cca975e96251a9e5ee', 'request_id': 'req-60093281-146c-4d49-8483-5c1a03e5895f'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:31.732083 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-60093281-146c-4d49-8483-5c1a03e5895f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3697365201b41cca975e96251a9e5ee', 'request_id': 'req-60093281-146c-4d49-8483-5c1a03e5895f'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:31.826131 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-60093281-146c-4d49-8483-5c1a03e5895f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3697365201b41cca975e96251a9e5ee', 'request_id': 'req-60093281-146c-4d49-8483-5c1a03e5895f'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:31.841675 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-60093281-146c-4d49-8483-5c1a03e5895f admin admin] The token cache is being invalidate because project d3697365201b41cca975e96251a9e5ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:57:31.842188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-60093281-146c-4d49-8483-5c1a03e5895f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:31.842948 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-60093281-146c-4d49-8483-5c1a03e5895f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:31.845080 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1302/2619] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:57:31 2022] DELETE /identity/v3/projects/d3697365201b41cca975e96251a9e5ee => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:31.849218 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be45f0-a6c6-4913-a9f7-181ca1ebff07 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:31.850881 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-08be45f0-a6c6-4913-a9f7-181ca1ebff07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:31.864865 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be45f0-a6c6-4913-a9f7-181ca1ebff07 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:31.925485 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431c983-fb55-4a56-aabc-5a217ff035fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:31.926371 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431c983-fb55-4a56-aabc-5a217ff035fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:31.926662 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431c983-fb55-4a56-aabc-5a217ff035fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:31.927007 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431c983-fb55-4a56-aabc-5a217ff035fe admin admin] PATH_INFO: `/v3/users/9a4caa4530114f14ad208acf74fc73fd` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:31.928820 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431c983-fb55-4a56-aabc-5a217ff035fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a4caa4530114f14ad208acf74fc73fd)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:31.930933 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431c983-fb55-4a56-aabc-5a217ff035fe admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:32.114407 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-7431c983-fb55-4a56-aabc-5a217ff035fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a4caa4530114f14ad208acf74fc73fd'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:32.126536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-7431c983-fb55-4a56-aabc-5a217ff035fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a4caa4530114f14ad208acf74fc73fd'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:32.203455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-7431c983-fb55-4a56-aabc-5a217ff035fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a4caa4530114f14ad208acf74fc73fd'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:32.204811 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-7431c983-fb55-4a56-aabc-5a217ff035fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a4caa4530114f14ad208acf74fc73fd'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:32.214869 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-7431c983-fb55-4a56-aabc-5a217ff035fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a4caa4530114f14ad208acf74fc73fd'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:32.222508 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1318/2620] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:57:31 2022] DELETE /identity/v3/users/9a4caa4530114f14ad208acf74fc73fd => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:32.657539 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60093281-146c-4d49-8483-5c1a03e5895f admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:32.659066 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-60093281-146c-4d49-8483-5c1a03e5895f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:32.675108 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60093281-146c-4d49-8483-5c1a03e5895f admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:32.735249 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d0e164-eca3-4b74-9fc0-c7f77c6710a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:32.736115 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0e164-eca3-4b74-9fc0-c7f77c6710a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:32.736398 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0e164-eca3-4b74-9fc0-c7f77c6710a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:32.736674 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0e164-eca3-4b74-9fc0-c7f77c6710a1 admin admin] PATH_INFO: `/v3/projects/fbca9f36903d49cba238b8eb2c171e26` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:32.737887 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0e164-eca3-4b74-9fc0-c7f77c6710a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbca9f36903d49cba238b8eb2c171e26)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:32.740035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d0e164-eca3-4b74-9fc0-c7f77c6710a1 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:32.854858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-44d0e164-eca3-4b74-9fc0-c7f77c6710a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbca9f36903d49cba238b8eb2c171e26', 'request_id': 'req-44d0e164-eca3-4b74-9fc0-c7f77c6710a1'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:32.861577 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-44d0e164-eca3-4b74-9fc0-c7f77c6710a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbca9f36903d49cba238b8eb2c171e26', 'request_id': 'req-44d0e164-eca3-4b74-9fc0-c7f77c6710a1'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:32.949263 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7dbff0-16bf-4d3d-b2ce-5e7cbaee8c59 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:32.949785 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7dbff0-16bf-4d3d-b2ce-5e7cbaee8c59 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:32.951360 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7dbff0-16bf-4d3d-b2ce-5e7cbaee8c59 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:32.951360 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1319/2621] 10.222.0.13 () {54 vars in 921 bytes} [Fri May 27 19:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:57:32.952154 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-44d0e164-eca3-4b74-9fc0-c7f77c6710a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbca9f36903d49cba238b8eb2c171e26', 'request_id': 'req-44d0e164-eca3-4b74-9fc0-c7f77c6710a1'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:32.958077 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06abf8d6-935a-4be8-81c5-4b5ecbb85534 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:32.958373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06abf8d6-935a-4be8-81c5-4b5ecbb85534 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:32.958655 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06abf8d6-935a-4be8-81c5-4b5ecbb85534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:32.966464 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-44d0e164-eca3-4b74-9fc0-c7f77c6710a1 admin admin] The token cache is being invalidate because project fbca9f36903d49cba238b8eb2c171e26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:57:32.966998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-44d0e164-eca3-4b74-9fc0-c7f77c6710a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:32.967631 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-44d0e164-eca3-4b74-9fc0-c7f77c6710a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:32.970322 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1303/2622] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:57:32 2022] DELETE /identity/v3/projects/fbca9f36903d49cba238b8eb2c171e26 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:32.975147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1462a99-8ad0-447b-8369-bce9e5b464da None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:32.979466 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1462a99-8ad0-447b-8369-bce9e5b464da None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:32.979466 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1462a99-8ad0-447b-8369-bce9e5b464da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:32.981867 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-06abf8d6-935a-4be8-81c5-4b5ecbb85534 None None] MFA Rules not processed for user `dd34f15400c4476bbb2161dbe1b8b2b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:32.983247 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-06abf8d6-935a-4be8-81c5-4b5ecbb85534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:32.996033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-c1462a99-8ad0-447b-8369-bce9e5b464da None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:33.001446 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-c1462a99-8ad0-447b-8369-bce9e5b464da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:33.007830 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:33.034035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:33.049555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1320/2623] 10.222.0.13 () {58 vars in 1018 bytes} [Fri May 27 19:57:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:33.062259 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06abf8d6-935a-4be8-81c5-4b5ecbb85534 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:33.063619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-06abf8d6-935a-4be8-81c5-4b5ecbb85534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:33.072220 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1304/2624] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:57:33.074544 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06abf8d6-935a-4be8-81c5-4b5ecbb85534 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:33.078722 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5065c-ef68-4cc0-8cc7-b3cfc89f27fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:33.079458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5065c-ef68-4cc0-8cc7-b3cfc89f27fd service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:33.079842 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5065c-ef68-4cc0-8cc7-b3cfc89f27fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:33.080040 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5065c-ef68-4cc0-8cc7-b3cfc89f27fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:33.080481 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06b0c2-5dd8-4663-bffb-aeee1d840782 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:33.080808 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06b0c2-5dd8-4663-bffb-aeee1d840782 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:33.081144 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06b0c2-5dd8-4663-bffb-aeee1d840782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:33.088645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5065c-ef68-4cc0-8cc7-b3cfc89f27fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:33.090129 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5065c-ef68-4cc0-8cc7-b3cfc89f27fd service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:33.098452 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1321/2625] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:57:33.101841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-ae06b0c2-5dd8-4663-bffb-aeee1d840782 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:33.103060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-ae06b0c2-5dd8-4663-bffb-aeee1d840782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:33.112400 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1305/2626] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:57:33.116944 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5065c-ef68-4cc0-8cc7-b3cfc89f27fd service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:33.128261 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5065c-ef68-4cc0-8cc7-b3cfc89f27fd service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:33.134096 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9feda6b-88c6-4f16-8d7c-306539be911d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:33.134760 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9feda6b-88c6-4f16-8d7c-306539be911d admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:33.135234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9feda6b-88c6-4f16-8d7c-306539be911d admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:33.136136 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9feda6b-88c6-4f16-8d7c-306539be911d admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:33.136926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9feda6b-88c6-4f16-8d7c-306539be911d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:57:33.137296 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9feda6b-88c6-4f16-8d7c-306539be911d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:33.138984 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9feda6b-88c6-4f16-8d7c-306539be911d admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:33.153745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1322/2627] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:57:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:57:33.156201 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06b0c2-5dd8-4663-bffb-aeee1d840782 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:33.166847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06b0c2-5dd8-4663-bffb-aeee1d840782 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:33.171959 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37275172-4cfa-45e2-bb63-7619a975ac04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:33.172659 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37275172-4cfa-45e2-bb63-7619a975ac04 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:33.173002 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37275172-4cfa-45e2-bb63-7619a975ac04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:33.173300 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37275172-4cfa-45e2-bb63-7619a975ac04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:33.174977 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37275172-4cfa-45e2-bb63-7619a975ac04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:33.176448 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37275172-4cfa-45e2-bb63-7619a975ac04 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:33.182139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77095e-5a05-42c3-b393-0f57d85c9ad8 None None] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:33.182429 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77095e-5a05-42c3-b393-0f57d85c9ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:33.182739 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a77095e-5a05-42c3-b393-0f57d85c9ad8 None None] PATH_INFO: `/` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:33.184299 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1323/2628] 10.222.0.13 () {54 vars in 921 bytes} [Fri May 27 19:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 19:57:33.189906 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7caad2-987b-43b7-98a2-300ee0daa886 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:33.190280 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7caad2-987b-43b7-98a2-300ee0daa886 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:33.190544 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7caad2-987b-43b7-98a2-300ee0daa886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:33.209291 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-5f7caad2-987b-43b7-98a2-300ee0daa886 None None] MFA Rules not processed for user `7542e1fd160e4b2c9d25accf19aad19f`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:33.211450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-5f7caad2-987b-43b7-98a2-300ee0daa886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:33.280377 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1324/2629] 10.222.0.13 () {58 vars in 1018 bytes} [Fri May 27 19:57:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:57:33.281266 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7caad2-987b-43b7-98a2-300ee0daa886 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:33.282619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-5f7caad2-987b-43b7-98a2-300ee0daa886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:33.292556 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1306/2630] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:57:33 2022] POST /identity/v3/projects => generated 379 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:33.293753 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37275172-4cfa-45e2-bb63-7619a975ac04 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:33.300434 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7caad2-987b-43b7-98a2-300ee0daa886 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:33.349944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37275172-4cfa-45e2-bb63-7619a975ac04 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:33.353734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f626aaf-f4df-4eef-8cea-228c1cf203e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:33.354411 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f626aaf-f4df-4eef-8cea-228c1cf203e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:33.354694 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f626aaf-f4df-4eef-8cea-228c1cf203e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:33.354972 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f626aaf-f4df-4eef-8cea-228c1cf203e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:33.358428 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e865ccc-4445-403e-8da2-1d4a9e38a1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:33.359307 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e865ccc-4445-403e-8da2-1d4a9e38a1a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:33.359591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e865ccc-4445-403e-8da2-1d4a9e38a1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:33.359872 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e865ccc-4445-403e-8da2-1d4a9e38a1a6 admin admin] PATH_INFO: `/v3/users/c97b80f94f424a2ca5622e066276515a` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:33.361710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e865ccc-4445-403e-8da2-1d4a9e38a1a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c97b80f94f424a2ca5622e066276515a)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:33.364475 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e865ccc-4445-403e-8da2-1d4a9e38a1a6 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:33.425213 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f626aaf-f4df-4eef-8cea-228c1cf203e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:33.426687 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f626aaf-f4df-4eef-8cea-228c1cf203e7 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:33.438602 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1307/2631] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:57:33.439384 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f626aaf-f4df-4eef-8cea-228c1cf203e7 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:33.449692 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f626aaf-f4df-4eef-8cea-228c1cf203e7 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:33.454797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a9195-762a-419e-8ca7-4b41f3c75e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:33.455522 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a9195-762a-419e-8ca7-4b41f3c75e9f admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:33.455807 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a9195-762a-419e-8ca7-4b41f3c75e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:33.456084 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a9195-762a-419e-8ca7-4b41f3c75e9f admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:33.456798 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a9195-762a-419e-8ca7-4b41f3c75e9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:33.458967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a9195-762a-419e-8ca7-4b41f3c75e9f admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:33.553973 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:33.565046 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-1e865ccc-4445-403e-8da2-1d4a9e38a1a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c97b80f94f424a2ca5622e066276515a'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:33.580753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-1e865ccc-4445-403e-8da2-1d4a9e38a1a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c97b80f94f424a2ca5622e066276515a'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:33.598119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1308/2632] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:57:33 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:33.598119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a9195-762a-419e-8ca7-4b41f3c75e9f admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:33.604850 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a9195-762a-419e-8ca7-4b41f3c75e9f admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:33.610824 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636542f-f3c7-4140-805b-9328cef4eeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:33.611576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636542f-f3c7-4140-805b-9328cef4eeb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:33.612733 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636542f-f3c7-4140-805b-9328cef4eeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:33.613281 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636542f-f3c7-4140-805b-9328cef4eeb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:33.613964 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636542f-f3c7-4140-805b-9328cef4eeb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:33.616373 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2636542f-f3c7-4140-805b-9328cef4eeb2 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:33.629455 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1309/2633] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:57:33.640231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636542f-f3c7-4140-805b-9328cef4eeb2 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:33.652338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636542f-f3c7-4140-805b-9328cef4eeb2 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:33.655294 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-1e865ccc-4445-403e-8da2-1d4a9e38a1a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c97b80f94f424a2ca5622e066276515a'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:33.655970 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-1e865ccc-4445-403e-8da2-1d4a9e38a1a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c97b80f94f424a2ca5622e066276515a'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:33.657847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713da8e2-e08b-4b6a-b52b-d7625daa5c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:33.658622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713da8e2-e08b-4b6a-b52b-d7625daa5c63 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:33.658969 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713da8e2-e08b-4b6a-b52b-d7625daa5c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:33.659349 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713da8e2-e08b-4b6a-b52b-d7625daa5c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:33.671516 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713da8e2-e08b-4b6a-b52b-d7625daa5c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:33.671516 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713da8e2-e08b-4b6a-b52b-d7625daa5c63 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:33.672007 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-1e865ccc-4445-403e-8da2-1d4a9e38a1a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c97b80f94f424a2ca5622e066276515a'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:33.675211 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1310/2634] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 19:57:33.676013 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1325/2635] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:57:33 2022] DELETE /identity/v3/users/c97b80f94f424a2ca5622e066276515a => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:33.682101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713da8e2-e08b-4b6a-b52b-d7625daa5c63 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:33.682572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe7c7b-80c9-47b9-a6a3-e2846286f5e4 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:33.683230 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe7c7b-80c9-47b9-a6a3-e2846286f5e4 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:33.683676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe7c7b-80c9-47b9-a6a3-e2846286f5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:33.685996 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-713da8e2-e08b-4b6a-b52b-d7625daa5c63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:33.702922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-1efe7c7b-80c9-47b9-a6a3-e2846286f5e4 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:33.704364 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-1efe7c7b-80c9-47b9-a6a3-e2846286f5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:33.705307 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713da8e2-e08b-4b6a-b52b-d7625daa5c63 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:33.711099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:33.776196 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1326/2636] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:33.785077 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b65a9a-db0c-4efe-b4b5-097823879635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:33.785427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe7c7b-80c9-47b9-a6a3-e2846286f5e4 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:33.786117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b65a9a-db0c-4efe-b4b5-097823879635 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:33.786507 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b65a9a-db0c-4efe-b4b5-097823879635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:33.786863 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-1efe7c7b-80c9-47b9-a6a3-e2846286f5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:33.787786 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b65a9a-db0c-4efe-b4b5-097823879635 admin admin] PATH_INFO: `/v3/projects/c67389355bb8440a9fc41dd2f07a7bac/users/2ad86f4f230a41aeb076a605b7517703/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:33.808299 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b65a9a-db0c-4efe-b4b5-097823879635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c67389355bb8440a9fc41dd2f07a7bac, user_id=2ad86f4f230a41aeb076a605b7517703, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:33.811026 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b65a9a-db0c-4efe-b4b5-097823879635 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:33.854710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efe7c7b-80c9-47b9-a6a3-e2846286f5e4 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:33.859261 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579618d3-3355-43a1-b726-b7b95bcfd3f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:33.860058 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579618d3-3355-43a1-b726-b7b95bcfd3f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:33.860347 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579618d3-3355-43a1-b726-b7b95bcfd3f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:33.860671 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579618d3-3355-43a1-b726-b7b95bcfd3f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:33.871626 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579618d3-3355-43a1-b726-b7b95bcfd3f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:33.873409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579618d3-3355-43a1-b726-b7b95bcfd3f7 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:33.884482 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1327/2637] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:57:33.902436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1311/2638] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:33 2022] PUT /identity/v3/projects/c67389355bb8440a9fc41dd2f07a7bac/users/2ad86f4f230a41aeb076a605b7517703/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:57:33.916057 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd5d0a-dcd1-4503-bf30-e15818251bf7 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:33.916365 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd5d0a-dcd1-4503-bf30-e15818251bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:33.916619 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd5d0a-dcd1-4503-bf30-e15818251bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:33.951352 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-a8cd5d0a-dcd1-4503-bf30-e15818251bf7 None None] MFA Rules not processed for user `2ad86f4f230a41aeb076a605b7517703`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:33.952585 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-a8cd5d0a-dcd1-4503-bf30-e15818251bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:34.004945 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:34.026438 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1328/2639] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:34.031087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b65a9a-db0c-4efe-b4b5-097823879635 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:34.107248 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b65a9a-db0c-4efe-b4b5-097823879635 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:34.112356 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a4e1c-5d73-4b21-b86b-a2933e735c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:34.113023 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a4e1c-5d73-4b21-b86b-a2933e735c72 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:34.113316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a4e1c-5d73-4b21-b86b-a2933e735c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:34.113607 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a4e1c-5d73-4b21-b86b-a2933e735c72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:34.114340 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a4e1c-5d73-4b21-b86b-a2933e735c72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:34.116528 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a4e1c-5d73-4b21-b86b-a2933e735c72 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:34.285471 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1312/2640] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:57:34 2022] POST /identity/v3/projects => generated 379 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:34.291630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd5d0a-dcd1-4503-bf30-e15818251bf7 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:34.308145 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd5d0a-dcd1-4503-bf30-e15818251bf7 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:34.377355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bb786-e648-4f8f-9e4e-e6f3fc89254c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:34.378044 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bb786-e648-4f8f-9e4e-e6f3fc89254c admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:34.378531 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bb786-e648-4f8f-9e4e-e6f3fc89254c admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:34.378662 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7bb786-e648-4f8f-9e4e-e6f3fc89254c admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:34.379359 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7bb786-e648-4f8f-9e4e-e6f3fc89254c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:34.381527 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7bb786-e648-4f8f-9e4e-e6f3fc89254c admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:34.469168 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1329/2641] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:57:34 2022] POST /identity/v3/users => generated 296 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:34.473548 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a4e1c-5d73-4b21-b86b-a2933e735c72 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:34.486810 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a4e1c-5d73-4b21-b86b-a2933e735c72 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:34.493273 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3911bdd-97d7-41c1-86d4-5e7f09f1614c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:34.494099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3911bdd-97d7-41c1-86d4-5e7f09f1614c admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:34.494467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3911bdd-97d7-41c1-86d4-5e7f09f1614c admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:34.494824 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3911bdd-97d7-41c1-86d4-5e7f09f1614c admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:34.495637 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3911bdd-97d7-41c1-86d4-5e7f09f1614c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:34.497430 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3911bdd-97d7-41c1-86d4-5e7f09f1614c admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:34.508455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bb786-e648-4f8f-9e4e-e6f3fc89254c admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:34.511934 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-7c7bb786-e648-4f8f-9e4e-e6f3fc89254c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:34.529598 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1313/2642] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:34.541383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3911bdd-97d7-41c1-86d4-5e7f09f1614c admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:34.546369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7bb786-e648-4f8f-9e4e-e6f3fc89254c admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:34.554489 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad191f-aa66-405c-8e38-819a80c09512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:34.555766 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad191f-aa66-405c-8e38-819a80c09512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:34.556149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad191f-aa66-405c-8e38-819a80c09512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:34.556545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad191f-aa66-405c-8e38-819a80c09512 admin admin] PATH_INFO: `/v3/projects/6c7809ae5cde4920a3473efcb60d2f22` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:34.558166 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad191f-aa66-405c-8e38-819a80c09512 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c7809ae5cde4920a3473efcb60d2f22)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:34.561354 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad191f-aa66-405c-8e38-819a80c09512 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:34.562938 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3911bdd-97d7-41c1-86d4-5e7f09f1614c admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:34.575994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42bcd7-cb86-4b12-be51-cef9768f4453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:34.579533 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42bcd7-cb86-4b12-be51-cef9768f4453 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:34.579836 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42bcd7-cb86-4b12-be51-cef9768f4453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:34.580132 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e42bcd7-cb86-4b12-be51-cef9768f4453 admin admin] PATH_INFO: `/v3/projects/7dbe70f446fd40638ce81902af391f0b/users/92affaa1686e4ac4a08a63da41e0036b/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:34.610964 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e42bcd7-cb86-4b12-be51-cef9768f4453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dbe70f446fd40638ce81902af391f0b, user_id=92affaa1686e4ac4a08a63da41e0036b, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:34.613719 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e42bcd7-cb86-4b12-be51-cef9768f4453 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:34.713737 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1314/2643] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:34 2022] PUT /identity/v3/projects/7dbe70f446fd40638ce81902af391f0b/users/92affaa1686e4ac4a08a63da41e0036b/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:34.719467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42bcd7-cb86-4b12-be51-cef9768f4453 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:34.734220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f5ad191f-aa66-405c-8e38-819a80c09512 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c7809ae5cde4920a3473efcb60d2f22', 'request_id': 'req-f5ad191f-aa66-405c-8e38-819a80c09512'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:34.744944 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f5ad191f-aa66-405c-8e38-819a80c09512 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c7809ae5cde4920a3473efcb60d2f22', 'request_id': 'req-f5ad191f-aa66-405c-8e38-819a80c09512'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:34.755914 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f5ad191f-aa66-405c-8e38-819a80c09512 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c7809ae5cde4920a3473efcb60d2f22', 'request_id': 'req-f5ad191f-aa66-405c-8e38-819a80c09512'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:34.802076 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:34.826012 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e42bcd7-cb86-4b12-be51-cef9768f4453 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:34.840904 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f5ad191f-aa66-405c-8e38-819a80c09512 admin admin] The token cache is being invalidate because project 6c7809ae5cde4920a3473efcb60d2f22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:57:34.841260 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f5ad191f-aa66-405c-8e38-819a80c09512 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:34.841909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f5ad191f-aa66-405c-8e38-819a80c09512 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:34.849573 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1330/2644] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:57:34 2022] DELETE /identity/v3/projects/6c7809ae5cde4920a3473efcb60d2f22 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:34.906264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9cb976-aca8-4d61-af0e-47e9c48763e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:34.907154 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9cb976-aca8-4d61-af0e-47e9c48763e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:34.907452 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9cb976-aca8-4d61-af0e-47e9c48763e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:34.908033 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9cb976-aca8-4d61-af0e-47e9c48763e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:34.909527 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9cb976-aca8-4d61-af0e-47e9c48763e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:34.911660 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9cb976-aca8-4d61-af0e-47e9c48763e6 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:34.950970 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1315/2645] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:34.951883 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad191f-aa66-405c-8e38-819a80c09512 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:34.953789 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-f5ad191f-aa66-405c-8e38-819a80c09512 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:34.971867 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad191f-aa66-405c-8e38-819a80c09512 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:34.978838 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8cc98-b764-4aa7-ad8d-44873f58b90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:34.979762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8cc98-b764-4aa7-ad8d-44873f58b90c admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:34.980052 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8cc98-b764-4aa7-ad8d-44873f58b90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:34.980353 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f8cc98-b764-4aa7-ad8d-44873f58b90c admin admin] PATH_INFO: `/v3/projects/7dbe70f446fd40638ce81902af391f0b/users/92affaa1686e4ac4a08a63da41e0036b/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:34.983066 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8cc98-b764-4aa7-ad8d-44873f58b90c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7dbe70f446fd40638ce81902af391f0b, user_id=92affaa1686e4ac4a08a63da41e0036b, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:34.987445 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f8cc98-b764-4aa7-ad8d-44873f58b90c admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:35.009445 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1331/2646] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:34 2022] PUT /identity/v3/projects/7dbe70f446fd40638ce81902af391f0b/users/92affaa1686e4ac4a08a63da41e0036b/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:35.015688 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebdc5d7-6637-438c-aa98-4d9a3c31ea5f None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:35.016028 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebdc5d7-6637-438c-aa98-4d9a3c31ea5f None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:35.016312 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebdc5d7-6637-438c-aa98-4d9a3c31ea5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:35.059696 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-1ebdc5d7-6637-438c-aa98-4d9a3c31ea5f None None] MFA Rules not processed for user `92affaa1686e4ac4a08a63da41e0036b`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:35.060982 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-1ebdc5d7-6637-438c-aa98-4d9a3c31ea5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:35.147893 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1316/2647] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:35 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:35.159317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8cc98-b764-4aa7-ad8d-44873f58b90c admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:35.161123 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-b2f8cc98-b764-4aa7-ad8d-44873f58b90c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:35.230028 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:35.234667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f8cc98-b764-4aa7-ad8d-44873f58b90c admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:35.238919 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ccab1-38d5-4fa5-b281-c6220cc748dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:35.239612 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ccab1-38d5-4fa5-b281-c6220cc748dc service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:35.239901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ccab1-38d5-4fa5-b281-c6220cc748dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:35.242743 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ccab1-38d5-4fa5-b281-c6220cc748dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:35.250124 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ccab1-38d5-4fa5-b281-c6220cc748dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:35.251580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ccab1-38d5-4fa5-b281-c6220cc748dc service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:35.265764 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1332/2648] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:35 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:57:35.831710 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebdc5d7-6637-438c-aa98-4d9a3c31ea5f None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:35.913356 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebdc5d7-6637-438c-aa98-4d9a3c31ea5f None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:35.922401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c35a4-ab4d-468f-b42a-be73a0c23e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:35.924219 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c35a4-ab4d-468f-b42a-be73a0c23e78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:35.924404 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c35a4-ab4d-468f-b42a-be73a0c23e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:35.924866 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c35a4-ab4d-468f-b42a-be73a0c23e78 admin admin] PATH_INFO: `/v3/users/2ad86f4f230a41aeb076a605b7517703` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:35.928707 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c35a4-ab4d-468f-b42a-be73a0c23e78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ad86f4f230a41aeb076a605b7517703)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:35.929336 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c35a4-ab4d-468f-b42a-be73a0c23e78 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:35.942704 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=100 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:36.143955 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-fa4c35a4-ab4d-468f-b42a-be73a0c23e78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ad86f4f230a41aeb076a605b7517703'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:36.156603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-fa4c35a4-ab4d-468f-b42a-be73a0c23e78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ad86f4f230a41aeb076a605b7517703'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:36.225899 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-fa4c35a4-ab4d-468f-b42a-be73a0c23e78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ad86f4f230a41aeb076a605b7517703'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:36.226581 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-fa4c35a4-ab4d-468f-b42a-be73a0c23e78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ad86f4f230a41aeb076a605b7517703'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:36.227240 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-fa4c35a4-ab4d-468f-b42a-be73a0c23e78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ad86f4f230a41aeb076a605b7517703'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:36.240998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1317/2649] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:57:35 2022] DELETE /identity/v3/users/2ad86f4f230a41aeb076a605b7517703 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:36.245945 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17c4f9-b316-491f-abe7-9369f8eff525 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:36.246443 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17c4f9-b316-491f-abe7-9369f8eff525 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:36.246761 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17c4f9-b316-491f-abe7-9369f8eff525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:36.272070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-fc17c4f9-b316-491f-abe7-9369f8eff525 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:36.273050 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-fc17c4f9-b316-491f-abe7-9369f8eff525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:36.351261 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1333/2650] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:36.360286 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c35a4-ab4d-468f-b42a-be73a0c23e78 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:36.361714 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-fa4c35a4-ab4d-468f-b42a-be73a0c23e78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:36.419095 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c35a4-ab4d-468f-b42a-be73a0c23e78 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:36.422669 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7d5b8-e3d2-42ca-bc84-ceb1190b7747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:36.423329 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7d5b8-e3d2-42ca-bc84-ceb1190b7747 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:36.423617 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7d5b8-e3d2-42ca-bc84-ceb1190b7747 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:36.423897 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7d5b8-e3d2-42ca-bc84-ceb1190b7747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:36.433804 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7d5b8-e3d2-42ca-bc84-ceb1190b7747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:36.435247 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7d5b8-e3d2-42ca-bc84-ceb1190b7747 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:36.444799 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1318/2651] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:57:37.016845 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17c4f9-b316-491f-abe7-9369f8eff525 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:37.018269 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-fc17c4f9-b316-491f-abe7-9369f8eff525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:37.031980 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17c4f9-b316-491f-abe7-9369f8eff525 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:37.037144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1703cc-b16c-420d-be0e-2ad3fe226183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:37.038033 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1703cc-b16c-420d-be0e-2ad3fe226183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:37.038338 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1703cc-b16c-420d-be0e-2ad3fe226183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:37.038633 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1703cc-b16c-420d-be0e-2ad3fe226183 admin admin] PATH_INFO: `/v3/projects/c67389355bb8440a9fc41dd2f07a7bac` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:37.039847 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1703cc-b16c-420d-be0e-2ad3fe226183 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c67389355bb8440a9fc41dd2f07a7bac)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:37.041855 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1703cc-b16c-420d-be0e-2ad3fe226183 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:37.152220 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-6d1703cc-b16c-420d-be0e-2ad3fe226183 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c67389355bb8440a9fc41dd2f07a7bac', 'request_id': 'req-6d1703cc-b16c-420d-be0e-2ad3fe226183'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:37.157806 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-6d1703cc-b16c-420d-be0e-2ad3fe226183 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c67389355bb8440a9fc41dd2f07a7bac', 'request_id': 'req-6d1703cc-b16c-420d-be0e-2ad3fe226183'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:37.163812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-6d1703cc-b16c-420d-be0e-2ad3fe226183 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c67389355bb8440a9fc41dd2f07a7bac', 'request_id': 'req-6d1703cc-b16c-420d-be0e-2ad3fe226183'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:37.260163 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-6d1703cc-b16c-420d-be0e-2ad3fe226183 admin admin] The token cache is being invalidate because project c67389355bb8440a9fc41dd2f07a7bac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:57:37.260791 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-6d1703cc-b16c-420d-be0e-2ad3fe226183 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:37.261614 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-6d1703cc-b16c-420d-be0e-2ad3fe226183 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:37.263254 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1334/2652] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:57:37 2022] DELETE /identity/v3/projects/c67389355bb8440a9fc41dd2f07a7bac => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:37.267437 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7d5b8-e3d2-42ca-bc84-ceb1190b7747 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:37.277627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7d5b8-e3d2-42ca-bc84-ceb1190b7747 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:37.335675 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b371436-b373-4da5-8123-c3a4b93e7420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:37.336468 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b371436-b373-4da5-8123-c3a4b93e7420 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:37.336755 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b371436-b373-4da5-8123-c3a4b93e7420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:37.337070 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b371436-b373-4da5-8123-c3a4b93e7420 admin admin] PATH_INFO: `/v3/users/92affaa1686e4ac4a08a63da41e0036b` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:37.338324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b371436-b373-4da5-8123-c3a4b93e7420 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92affaa1686e4ac4a08a63da41e0036b)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:37.340293 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b371436-b373-4da5-8123-c3a4b93e7420 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:37.518839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:37.532823 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0b371436-b373-4da5-8123-c3a4b93e7420 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92affaa1686e4ac4a08a63da41e0036b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:37.545604 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0b371436-b373-4da5-8123-c3a4b93e7420 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92affaa1686e4ac4a08a63da41e0036b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:37.614138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0b371436-b373-4da5-8123-c3a4b93e7420 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92affaa1686e4ac4a08a63da41e0036b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:37.614761 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0b371436-b373-4da5-8123-c3a4b93e7420 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92affaa1686e4ac4a08a63da41e0036b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:37.615934 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-0b371436-b373-4da5-8123-c3a4b93e7420 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92affaa1686e4ac4a08a63da41e0036b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:37.629433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1319/2653] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:57:37 2022] DELETE /identity/v3/users/92affaa1686e4ac4a08a63da41e0036b => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:38.119189 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1703cc-b16c-420d-be0e-2ad3fe226183 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:38.122617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-6d1703cc-b16c-420d-be0e-2ad3fe226183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:38.135586 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1703cc-b16c-420d-be0e-2ad3fe226183 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:38.198938 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb90cf6-2fda-4963-a759-28a319a6bc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:38.199805 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb90cf6-2fda-4963-a759-28a319a6bc11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:38.200089 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb90cf6-2fda-4963-a759-28a319a6bc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:38.200372 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb90cf6-2fda-4963-a759-28a319a6bc11 admin admin] PATH_INFO: `/v3/projects/7dbe70f446fd40638ce81902af391f0b` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:38.201607 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb90cf6-2fda-4963-a759-28a319a6bc11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7dbe70f446fd40638ce81902af391f0b)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:38.203523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb90cf6-2fda-4963-a759-28a319a6bc11 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:38.220207 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:38.317630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-0cb90cf6-2fda-4963-a759-28a319a6bc11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7dbe70f446fd40638ce81902af391f0b', 'request_id': 'req-0cb90cf6-2fda-4963-a759-28a319a6bc11'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:38.323883 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-0cb90cf6-2fda-4963-a759-28a319a6bc11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7dbe70f446fd40638ce81902af391f0b', 'request_id': 'req-0cb90cf6-2fda-4963-a759-28a319a6bc11'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:38.330022 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-0cb90cf6-2fda-4963-a759-28a319a6bc11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7dbe70f446fd40638ce81902af391f0b', 'request_id': 'req-0cb90cf6-2fda-4963-a759-28a319a6bc11'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:38.396495 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-0cb90cf6-2fda-4963-a759-28a319a6bc11 admin admin] The token cache is being invalidate because project 7dbe70f446fd40638ce81902af391f0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:57:38.396714 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-0cb90cf6-2fda-4963-a759-28a319a6bc11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:38.397370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-0cb90cf6-2fda-4963-a759-28a319a6bc11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:38.399899 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1335/2654] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:57:38 2022] DELETE /identity/v3/projects/7dbe70f446fd40638ce81902af391f0b => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:38.404788 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340f490-9898-4cc0-a429-face52b688df None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:38.405285 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340f490-9898-4cc0-a429-face52b688df None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:38.405402 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f340f490-9898-4cc0-a429-face52b688df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:38.425710 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-f340f490-9898-4cc0-a429-face52b688df None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:38.427092 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-f340f490-9898-4cc0-a429-face52b688df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:38.488112 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1320/2655] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:38.496297 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb18fa8-aa3e-46db-bf99-dd370405a977 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:38.496624 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb18fa8-aa3e-46db-bf99-dd370405a977 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:38.496906 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb18fa8-aa3e-46db-bf99-dd370405a977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:38.516994 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-ffb18fa8-aa3e-46db-bf99-dd370405a977 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:38.518207 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ffb18fa8-aa3e-46db-bf99-dd370405a977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:38.526964 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1336/2656] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:38.531438 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f340f490-9898-4cc0-a429-face52b688df None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:38.541172 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f340f490-9898-4cc0-a429-face52b688df None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:38.545711 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a482a9-37da-431b-91ab-ebd7d7b51607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:38.546364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a482a9-37da-431b-91ab-ebd7d7b51607 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:38.546644 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a482a9-37da-431b-91ab-ebd7d7b51607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:38.546918 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a482a9-37da-431b-91ab-ebd7d7b51607 admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:38.547644 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a482a9-37da-431b-91ab-ebd7d7b51607 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:57:38.547946 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a482a9-37da-431b-91ab-ebd7d7b51607 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:38.549560 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a482a9-37da-431b-91ab-ebd7d7b51607 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:38.560906 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1321/2657] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:57:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:57:38.565223 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb18fa8-aa3e-46db-bf99-dd370405a977 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:38.575502 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb18fa8-aa3e-46db-bf99-dd370405a977 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:38.580197 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a92d9-a384-473f-940d-d36f318e0efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:38.580850 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a92d9-a384-473f-940d-d36f318e0efa admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:38.581169 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a92d9-a384-473f-940d-d36f318e0efa admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:38.581451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70a92d9-a384-473f-940d-d36f318e0efa admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:38.582244 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a92d9-a384-473f-940d-d36f318e0efa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:38.584060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70a92d9-a384-473f-940d-d36f318e0efa admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:38.667481 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1337/2658] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:57:38 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:38.671571 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a482a9-37da-431b-91ab-ebd7d7b51607 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:38.682040 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a482a9-37da-431b-91ab-ebd7d7b51607 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:38.737976 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481e6ea-ea8c-4e3c-8fe8-c4f41627901b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:38.738645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481e6ea-ea8c-4e3c-8fe8-c4f41627901b admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:38.738929 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481e6ea-ea8c-4e3c-8fe8-c4f41627901b admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:38.739230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481e6ea-ea8c-4e3c-8fe8-c4f41627901b admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:38.739909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481e6ea-ea8c-4e3c-8fe8-c4f41627901b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:38.742470 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1481e6ea-ea8c-4e3c-8fe8-c4f41627901b admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:38.814761 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1322/2659] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:57:38 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:38.818674 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a92d9-a384-473f-940d-d36f318e0efa admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:38.829630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70a92d9-a384-473f-940d-d36f318e0efa admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:38.834439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04df512-7025-4b7a-b9a6-9c95fe99a67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:38.835837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04df512-7025-4b7a-b9a6-9c95fe99a67f admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:38.836315 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04df512-7025-4b7a-b9a6-9c95fe99a67f admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:38.836592 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04df512-7025-4b7a-b9a6-9c95fe99a67f admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:38.837354 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04df512-7025-4b7a-b9a6-9c95fe99a67f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:38.839012 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04df512-7025-4b7a-b9a6-9c95fe99a67f admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:38.854962 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1338/2660] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:38.859644 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481e6ea-ea8c-4e3c-8fe8-c4f41627901b admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:38.870273 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481e6ea-ea8c-4e3c-8fe8-c4f41627901b admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:38.874881 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a7ee6-0a73-4572-b8c0-0285528b7ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:38.875548 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a7ee6-0a73-4572-b8c0-0285528b7ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:38.875838 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a7ee6-0a73-4572-b8c0-0285528b7ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:38.876116 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a7ee6-0a73-4572-b8c0-0285528b7ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:38.876844 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a7ee6-0a73-4572-b8c0-0285528b7ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:38.878473 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a7ee6-0a73-4572-b8c0-0285528b7ce0 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:38.888977 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1323/2661] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:38.894205 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04df512-7025-4b7a-b9a6-9c95fe99a67f admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:38.905604 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04df512-7025-4b7a-b9a6-9c95fe99a67f admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:38.910509 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761c1cc-9cdf-4c6d-8ffb-c1315b3e6a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:38.911460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761c1cc-9cdf-4c6d-8ffb-c1315b3e6a0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:38.911809 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761c1cc-9cdf-4c6d-8ffb-c1315b3e6a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:38.912096 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b761c1cc-9cdf-4c6d-8ffb-c1315b3e6a0c admin admin] PATH_INFO: `/v3/projects/f2c0aedd5cc44f13853822c762a9b1cc/users/c093fbfc192a47919aaa7949c429a911/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:38.928833 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b761c1cc-9cdf-4c6d-8ffb-c1315b3e6a0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2c0aedd5cc44f13853822c762a9b1cc, user_id=c093fbfc192a47919aaa7949c429a911, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:38.932006 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b761c1cc-9cdf-4c6d-8ffb-c1315b3e6a0c admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:39.026363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1339/2662] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:38 2022] PUT /identity/v3/projects/f2c0aedd5cc44f13853822c762a9b1cc/users/c093fbfc192a47919aaa7949c429a911/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:39.037496 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f74c5b-61b0-4f9d-8424-e9e5dab2cf45 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:39.037794 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f74c5b-61b0-4f9d-8424-e9e5dab2cf45 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:39.038080 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f74c5b-61b0-4f9d-8424-e9e5dab2cf45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:39.069451 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-b8f74c5b-61b0-4f9d-8424-e9e5dab2cf45 None None] MFA Rules not processed for user `c093fbfc192a47919aaa7949c429a911`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:39.070918 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-b8f74c5b-61b0-4f9d-8424-e9e5dab2cf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:39.076188 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:39.136689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1324/2663] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:39.142047 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761c1cc-9cdf-4c6d-8ffb-c1315b3e6a0c admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:39.211523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b761c1cc-9cdf-4c6d-8ffb-c1315b3e6a0c admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:39.216498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7123139-6204-4cec-94a6-a0081f038896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:39.217352 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7123139-6204-4cec-94a6-a0081f038896 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:39.217704 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7123139-6204-4cec-94a6-a0081f038896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:39.217988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7123139-6204-4cec-94a6-a0081f038896 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:39.218686 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7123139-6204-4cec-94a6-a0081f038896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:39.220599 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7123139-6204-4cec-94a6-a0081f038896 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:39.325151 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1340/2664] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:57:39 2022] POST /identity/v3/projects => generated 385 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:39.329275 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f74c5b-61b0-4f9d-8424-e9e5dab2cf45 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:39.342407 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f74c5b-61b0-4f9d-8424-e9e5dab2cf45 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:39.397238 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056d7e9-2ed4-4c93-9ff2-8c803c521626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:39.398649 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056d7e9-2ed4-4c93-9ff2-8c803c521626 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:39.398943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056d7e9-2ed4-4c93-9ff2-8c803c521626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:39.399339 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3056d7e9-2ed4-4c93-9ff2-8c803c521626 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:39.400153 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056d7e9-2ed4-4c93-9ff2-8c803c521626 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:39.402384 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3056d7e9-2ed4-4c93-9ff2-8c803c521626 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:39.494342 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1325/2665] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:57:39 2022] POST /identity/v3/users => generated 299 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:57:39.498318 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7123139-6204-4cec-94a6-a0081f038896 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:39.509474 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7123139-6204-4cec-94a6-a0081f038896 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:39.514441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871a707-bcdb-4f7c-8ad2-4226e09bd5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:39.515118 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871a707-bcdb-4f7c-8ad2-4226e09bd5b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:39.515405 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871a707-bcdb-4f7c-8ad2-4226e09bd5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:39.515688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5871a707-bcdb-4f7c-8ad2-4226e09bd5b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:39.516417 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5871a707-bcdb-4f7c-8ad2-4226e09bd5b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:39.518070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5871a707-bcdb-4f7c-8ad2-4226e09bd5b0 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:39.530282 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1341/2666] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:39.534425 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056d7e9-2ed4-4c93-9ff2-8c803c521626 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:39.544694 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3056d7e9-2ed4-4c93-9ff2-8c803c521626 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:39.549333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f362ddc-6fac-455f-8c34-d7da0f205711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:39.550271 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f362ddc-6fac-455f-8c34-d7da0f205711 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:39.550562 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f362ddc-6fac-455f-8c34-d7da0f205711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:39.550841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f362ddc-6fac-455f-8c34-d7da0f205711 admin admin] PATH_INFO: `/v3/projects/149c09fb51384cd89b62c1e1899b2f94/users/59152c9c8fcd417bbb0b77ada0416b8b/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:39.565866 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f362ddc-6fac-455f-8c34-d7da0f205711 admin admin] RBAC: Authorizing `identity:create_grant(project_id=149c09fb51384cd89b62c1e1899b2f94, user_id=59152c9c8fcd417bbb0b77ada0416b8b, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:39.568512 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f362ddc-6fac-455f-8c34-d7da0f205711 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:39.642353 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1326/2667] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:39 2022] PUT /identity/v3/projects/149c09fb51384cd89b62c1e1899b2f94/users/59152c9c8fcd417bbb0b77ada0416b8b/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:39.646505 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871a707-bcdb-4f7c-8ad2-4226e09bd5b0 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:39.656927 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5871a707-bcdb-4f7c-8ad2-4226e09bd5b0 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:39.665842 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:39.713119 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab48b21-eb31-4f80-84bd-34513521ff45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:39.713782 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab48b21-eb31-4f80-84bd-34513521ff45 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:39.714065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab48b21-eb31-4f80-84bd-34513521ff45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:39.714338 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab48b21-eb31-4f80-84bd-34513521ff45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:39.715054 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab48b21-eb31-4f80-84bd-34513521ff45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:39.716698 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab48b21-eb31-4f80-84bd-34513521ff45 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:39.727208 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1342/2668] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:39.731451 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f362ddc-6fac-455f-8c34-d7da0f205711 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:39.744793 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f362ddc-6fac-455f-8c34-d7da0f205711 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:39.750894 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570decba-21da-4d4c-994f-6c542c6c5fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:39.751775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570decba-21da-4d4c-994f-6c542c6c5fa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:39.754329 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570decba-21da-4d4c-994f-6c542c6c5fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:39.754329 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570decba-21da-4d4c-994f-6c542c6c5fa3 admin admin] PATH_INFO: `/v3/projects/149c09fb51384cd89b62c1e1899b2f94/users/59152c9c8fcd417bbb0b77ada0416b8b/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:39.754329 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570decba-21da-4d4c-994f-6c542c6c5fa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=149c09fb51384cd89b62c1e1899b2f94, user_id=59152c9c8fcd417bbb0b77ada0416b8b, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:39.756943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570decba-21da-4d4c-994f-6c542c6c5fa3 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:39.768091 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1327/2669] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:39 2022] PUT /identity/v3/projects/149c09fb51384cd89b62c1e1899b2f94/users/59152c9c8fcd417bbb0b77ada0416b8b/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:39.778443 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf9e40-2f64-40f6-9d6a-efeb580379ec None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:39.778759 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf9e40-2f64-40f6-9d6a-efeb580379ec None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:39.783019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facf9e40-2f64-40f6-9d6a-efeb580379ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:39.806841 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-facf9e40-2f64-40f6-9d6a-efeb580379ec None None] MFA Rules not processed for user `59152c9c8fcd417bbb0b77ada0416b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:39.808072 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-facf9e40-2f64-40f6-9d6a-efeb580379ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:39.874817 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1343/2670] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:39 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:39.885962 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570decba-21da-4d4c-994f-6c542c6c5fa3 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:39.887707 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-570decba-21da-4d4c-994f-6c542c6c5fa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:39.945609 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570decba-21da-4d4c-994f-6c542c6c5fa3 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:39.949289 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d0bc4-0a88-483e-a7ab-f512ae5ba664 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:39.949944 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d0bc4-0a88-483e-a7ab-f512ae5ba664 service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:39.950225 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d0bc4-0a88-483e-a7ab-f512ae5ba664 service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:39.950501 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0d0bc4-0a88-483e-a7ab-f512ae5ba664 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:39.960202 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:40.006623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d0bc4-0a88-483e-a7ab-f512ae5ba664 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:40.008330 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0d0bc4-0a88-483e-a7ab-f512ae5ba664 service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:40.018294 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1328/2671] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:57:40.562897 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf9e40-2f64-40f6-9d6a-efeb580379ec None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:40.564432 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-facf9e40-2f64-40f6-9d6a-efeb580379ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:40.624231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facf9e40-2f64-40f6-9d6a-efeb580379ec None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:40.627822 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787855e0-8d76-484a-be77-0e2da96e683a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:40.628466 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787855e0-8d76-484a-be77-0e2da96e683a service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:40.628746 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787855e0-8d76-484a-be77-0e2da96e683a service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:40.629061 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787855e0-8d76-484a-be77-0e2da96e683a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:40.630910 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-787855e0-8d76-484a-be77-0e2da96e683a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:40.684069 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:40.701069 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787855e0-8d76-484a-be77-0e2da96e683a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:40.702579 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787855e0-8d76-484a-be77-0e2da96e683a service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:40.714316 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1344/2672] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:57:44.471994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d0bc4-0a88-483e-a7ab-f512ae5ba664 service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:44.473517 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-9e0d0bc4-0a88-483e-a7ab-f512ae5ba664 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:44.488801 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0d0bc4-0a88-483e-a7ab-f512ae5ba664 service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:44.494570 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d76c2b-9173-44f8-a16c-a7ef93caf62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:44.495480 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d76c2b-9173-44f8-a16c-a7ef93caf62d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:44.495771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d76c2b-9173-44f8-a16c-a7ef93caf62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:44.496057 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d76c2b-9173-44f8-a16c-a7ef93caf62d admin admin] PATH_INFO: `/v3/users/ed8754ddcfd54c24ac923db0fedc1606` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:44.497310 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d76c2b-9173-44f8-a16c-a7ef93caf62d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed8754ddcfd54c24ac923db0fedc1606)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:44.499245 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d76c2b-9173-44f8-a16c-a7ef93caf62d admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:44.726853 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f6d76c2b-9173-44f8-a16c-a7ef93caf62d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed8754ddcfd54c24ac923db0fedc1606'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:44.739839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f6d76c2b-9173-44f8-a16c-a7ef93caf62d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed8754ddcfd54c24ac923db0fedc1606'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:44.808634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f6d76c2b-9173-44f8-a16c-a7ef93caf62d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed8754ddcfd54c24ac923db0fedc1606'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:44.810816 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f6d76c2b-9173-44f8-a16c-a7ef93caf62d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed8754ddcfd54c24ac923db0fedc1606'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:44.811659 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f6d76c2b-9173-44f8-a16c-a7ef93caf62d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed8754ddcfd54c24ac923db0fedc1606'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:44.829160 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1329/2673] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:57:44 2022] DELETE /identity/v3/users/ed8754ddcfd54c24ac923db0fedc1606 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:45.148322 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787855e0-8d76-484a-be77-0e2da96e683a service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:45.167007 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787855e0-8d76-484a-be77-0e2da96e683a service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:45.168703 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d76c2b-9173-44f8-a16c-a7ef93caf62d admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:45.170084 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-f6d76c2b-9173-44f8-a16c-a7ef93caf62d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:45.233767 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af34258-10df-45fe-ab1b-c1126e7d7467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:45.234727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af34258-10df-45fe-ab1b-c1126e7d7467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:45.235082 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af34258-10df-45fe-ab1b-c1126e7d7467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:45.235435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af34258-10df-45fe-ab1b-c1126e7d7467 admin admin] PATH_INFO: `/v3/projects/2efe5f8c717640528624dc103744dcf4` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:45.236693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af34258-10df-45fe-ab1b-c1126e7d7467 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2efe5f8c717640528624dc103744dcf4)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:45.239481 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af34258-10df-45fe-ab1b-c1126e7d7467 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:45.254415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d76c2b-9173-44f8-a16c-a7ef93caf62d admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:45.258651 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d414317-9146-4600-8a5e-c597b2891ea3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:45.259329 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d414317-9146-4600-8a5e-c597b2891ea3 service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:45.259611 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d414317-9146-4600-8a5e-c597b2891ea3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:45.259885 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d414317-9146-4600-8a5e-c597b2891ea3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:45.261704 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-1d414317-9146-4600-8a5e-c597b2891ea3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:45.300829 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:45.324217 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d414317-9146-4600-8a5e-c597b2891ea3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:45.325686 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d414317-9146-4600-8a5e-c597b2891ea3 service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:45.335364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1330/2674] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:45 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:57:45.354428 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-9af34258-10df-45fe-ab1b-c1126e7d7467 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2efe5f8c717640528624dc103744dcf4', 'request_id': 'req-9af34258-10df-45fe-ab1b-c1126e7d7467'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:45.359862 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-9af34258-10df-45fe-ab1b-c1126e7d7467 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2efe5f8c717640528624dc103744dcf4', 'request_id': 'req-9af34258-10df-45fe-ab1b-c1126e7d7467'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:45.366487 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-9af34258-10df-45fe-ab1b-c1126e7d7467 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2efe5f8c717640528624dc103744dcf4', 'request_id': 'req-9af34258-10df-45fe-ab1b-c1126e7d7467'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:45.430439 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-9af34258-10df-45fe-ab1b-c1126e7d7467 admin admin] The token cache is being invalidate because project 2efe5f8c717640528624dc103744dcf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:57:45.433061 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-9af34258-10df-45fe-ab1b-c1126e7d7467 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:45.433061 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-9af34258-10df-45fe-ab1b-c1126e7d7467 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:45.433061 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1345/2675] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:57:45 2022] DELETE /identity/v3/projects/2efe5f8c717640528624dc103744dcf4 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:57:45.438151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b7338-f292-46f3-bbbd-19bcac6d3544 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:45.438878 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b7338-f292-46f3-bbbd-19bcac6d3544 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:45.438878 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b7338-f292-46f3-bbbd-19bcac6d3544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:45.458453 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-5f5b7338-f292-46f3-bbbd-19bcac6d3544 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:45.460482 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-5f5b7338-f292-46f3-bbbd-19bcac6d3544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:45.520439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1331/2676] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:57:45.529837 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9a63e-f5f2-43a3-a9e4-bcad11f220fb None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:45.530128 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9a63e-f5f2-43a3-a9e4-bcad11f220fb None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:45.530408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9a63e-f5f2-43a3-a9e4-bcad11f220fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:45.549110 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-daa9a63e-f5f2-43a3-a9e4-bcad11f220fb None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:45.550315 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-daa9a63e-f5f2-43a3-a9e4-bcad11f220fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:45.559343 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1346/2677] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:45.563628 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b7338-f292-46f3-bbbd-19bcac6d3544 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:45.573535 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b7338-f292-46f3-bbbd-19bcac6d3544 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:45.578291 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a65f0-ce1d-415e-a936-7b72b211c238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:45.578941 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a65f0-ce1d-415e-a936-7b72b211c238 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:45.579229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a65f0-ce1d-415e-a936-7b72b211c238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:45.579505 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39a65f0-ce1d-415e-a936-7b72b211c238 admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:45.580254 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a65f0-ce1d-415e-a936-7b72b211c238 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:57:45.580559 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a65f0-ce1d-415e-a936-7b72b211c238 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:45.582192 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39a65f0-ce1d-415e-a936-7b72b211c238 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:45.594353 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1332/2678] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:57:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:57:45.598611 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9a63e-f5f2-43a3-a9e4-bcad11f220fb None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:45.608766 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9a63e-f5f2-43a3-a9e4-bcad11f220fb None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:45.613603 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768088b-17ae-4607-b369-d9a0728b81c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:45.614252 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768088b-17ae-4607-b369-d9a0728b81c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:45.614537 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768088b-17ae-4607-b369-d9a0728b81c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:45.614816 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768088b-17ae-4607-b369-d9a0728b81c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:45.615498 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c768088b-17ae-4607-b369-d9a0728b81c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:45.617659 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c768088b-17ae-4607-b369-d9a0728b81c7 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:45.735236 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1347/2679] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:57:45 2022] POST /identity/v3/projects => generated 375 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:45.739528 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a65f0-ce1d-415e-a936-7b72b211c238 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:45.749922 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39a65f0-ce1d-415e-a936-7b72b211c238 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:45.807871 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f8348-3836-4a88-88e4-6015eba9bee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:45.808545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f8348-3836-4a88-88e4-6015eba9bee2 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:45.808831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f8348-3836-4a88-88e4-6015eba9bee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:45.809141 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f8348-3836-4a88-88e4-6015eba9bee2 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:45.809893 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f8348-3836-4a88-88e4-6015eba9bee2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:45.811759 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f8348-3836-4a88-88e4-6015eba9bee2 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:45.893882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1333/2680] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:57:45 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:57:45.898516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768088b-17ae-4607-b369-d9a0728b81c7 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:45.908731 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768088b-17ae-4607-b369-d9a0728b81c7 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:45.913495 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73e89b-7c32-40e4-9bfe-f5776b23e3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:45.914179 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73e89b-7c32-40e4-9bfe-f5776b23e3e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:45.914463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73e89b-7c32-40e4-9bfe-f5776b23e3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:45.914742 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d73e89b-7c32-40e4-9bfe-f5776b23e3e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:45.915531 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73e89b-7c32-40e4-9bfe-f5776b23e3e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:45.917162 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d73e89b-7c32-40e4-9bfe-f5776b23e3e2 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:45.927574 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1348/2681] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:45.931631 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f8348-3836-4a88-88e4-6015eba9bee2 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:45.941859 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f8348-3836-4a88-88e4-6015eba9bee2 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:45.947231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555cfd99-100d-4ee2-ad23-a49a48fde5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:45.947930 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cfd99-100d-4ee2-ad23-a49a48fde5ad admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:45.948215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cfd99-100d-4ee2-ad23-a49a48fde5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:45.948494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cfd99-100d-4ee2-ad23-a49a48fde5ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:45.949324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555cfd99-100d-4ee2-ad23-a49a48fde5ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:45.951064 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555cfd99-100d-4ee2-ad23-a49a48fde5ad admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:45.965925 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1334/2682] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:45.967176 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73e89b-7c32-40e4-9bfe-f5776b23e3e2 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:45.977934 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d73e89b-7c32-40e4-9bfe-f5776b23e3e2 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:45.982868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474dad2-b338-4a61-8397-b6403e459cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:45.989160 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474dad2-b338-4a61-8397-b6403e459cb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:45.989160 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474dad2-b338-4a61-8397-b6403e459cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:45.989160 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f474dad2-b338-4a61-8397-b6403e459cb0 admin admin] PATH_INFO: `/v3/projects/51d8222fe16e403999de6fc5803b50d3/users/bcdd3f30e42b41308b69e1233686cc17/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:46.001022 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474dad2-b338-4a61-8397-b6403e459cb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51d8222fe16e403999de6fc5803b50d3, user_id=bcdd3f30e42b41308b69e1233686cc17, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:46.003684 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f474dad2-b338-4a61-8397-b6403e459cb0 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:46.106356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1349/2683] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:45 2022] PUT /identity/v3/projects/51d8222fe16e403999de6fc5803b50d3/users/bcdd3f30e42b41308b69e1233686cc17/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:46.111409 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093972b3-6526-4a0b-8461-a9aa0812360e None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:46.111704 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093972b3-6526-4a0b-8461-a9aa0812360e None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:46.115542 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093972b3-6526-4a0b-8461-a9aa0812360e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:46.130573 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-093972b3-6526-4a0b-8461-a9aa0812360e None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:46.132698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-093972b3-6526-4a0b-8461-a9aa0812360e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:46.168583 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:46.194648 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1335/2684] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:57:46.204241 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474dad2-b338-4a61-8397-b6403e459cb0 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:46.206005 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-f474dad2-b338-4a61-8397-b6403e459cb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:46.222843 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:46.267549 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f474dad2-b338-4a61-8397-b6403e459cb0 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:46.271352 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebebc7dc-0bfe-44d5-a913-d599b99bea8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:46.271998 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebc7dc-0bfe-44d5-a913-d599b99bea8a service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:46.272349 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebc7dc-0bfe-44d5-a913-d599b99bea8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:46.273151 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebc7dc-0bfe-44d5-a913-d599b99bea8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:46.282867 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebebc7dc-0bfe-44d5-a913-d599b99bea8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:46.284405 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebebc7dc-0bfe-44d5-a913-d599b99bea8a service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:46.294202 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1350/2685] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:57:51.227991 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093972b3-6526-4a0b-8461-a9aa0812360e None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:51.229785 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-093972b3-6526-4a0b-8461-a9aa0812360e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:51.247318 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093972b3-6526-4a0b-8461-a9aa0812360e None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:51.256384 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46bd87-3618-4a64-a2ca-fdfbdcb0cb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:51.256384 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46bd87-3618-4a64-a2ca-fdfbdcb0cb87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:51.256384 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46bd87-3618-4a64-a2ca-fdfbdcb0cb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:51.256384 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b46bd87-3618-4a64-a2ca-fdfbdcb0cb87 admin admin] PATH_INFO: `/v3/users/c093fbfc192a47919aaa7949c429a911` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:51.256841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46bd87-3618-4a64-a2ca-fdfbdcb0cb87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c093fbfc192a47919aaa7949c429a911)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:51.258917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b46bd87-3618-4a64-a2ca-fdfbdcb0cb87 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:51.455956 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-4b46bd87-3618-4a64-a2ca-fdfbdcb0cb87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c093fbfc192a47919aaa7949c429a911'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:51.478532 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-4b46bd87-3618-4a64-a2ca-fdfbdcb0cb87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c093fbfc192a47919aaa7949c429a911'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:51.571269 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-4b46bd87-3618-4a64-a2ca-fdfbdcb0cb87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c093fbfc192a47919aaa7949c429a911'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:51.575385 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-4b46bd87-3618-4a64-a2ca-fdfbdcb0cb87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c093fbfc192a47919aaa7949c429a911'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:51.575385 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-4b46bd87-3618-4a64-a2ca-fdfbdcb0cb87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c093fbfc192a47919aaa7949c429a911'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:51.596916 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1336/2686] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:57:51 2022] DELETE /identity/v3/users/c093fbfc192a47919aaa7949c429a911 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:51.598702 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d681c-8172-4cb0-a4c6-7f7f4565fcde None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:51.599192 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d681c-8172-4cb0-a4c6-7f7f4565fcde None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:51.599386 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d681c-8172-4cb0-a4c6-7f7f4565fcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:51.620551 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-838d681c-8172-4cb0-a4c6-7f7f4565fcde None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:51.621793 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-838d681c-8172-4cb0-a4c6-7f7f4565fcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:51.703945 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1351/2687] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:57:51.718390 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46bd87-3618-4a64-a2ca-fdfbdcb0cb87 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:51.721641 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-4b46bd87-3618-4a64-a2ca-fdfbdcb0cb87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:51.722021 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4524c-d333-4713-98d8-0f17f6152aa8 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:51.722318 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4524c-d333-4713-98d8-0f17f6152aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:51.722605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4524c-d333-4713-98d8-0f17f6152aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:51.774755 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-d4e4524c-d333-4713-98d8-0f17f6152aa8 None None] MFA Rules not processed for user `bcdd3f30e42b41308b69e1233686cc17`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:51.776089 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d4e4524c-d333-4713-98d8-0f17f6152aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:51.799857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b46bd87-3618-4a64-a2ca-fdfbdcb0cb87 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:51.805130 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f27e2e-d2c8-45d9-a261-16aad5a4f2a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:51.805802 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f27e2e-d2c8-45d9-a261-16aad5a4f2a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:51.806087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f27e2e-d2c8-45d9-a261-16aad5a4f2a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:51.806366 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f27e2e-d2c8-45d9-a261-16aad5a4f2a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:51.816216 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f27e2e-d2c8-45d9-a261-16aad5a4f2a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:51.817705 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f27e2e-d2c8-45d9-a261-16aad5a4f2a2 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:51.828293 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1337/2688] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:57:51.859135 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1352/2689] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:51.876219 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f27e2e-d2c8-45d9-a261-16aad5a4f2a2 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:51.877926 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-22f27e2e-d2c8-45d9-a261-16aad5a4f2a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:51.940989 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:51.962351 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f27e2e-d2c8-45d9-a261-16aad5a4f2a2 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:51.967975 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafbec0-5255-40f3-a258-07a9d951563a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:51.968629 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafbec0-5255-40f3-a258-07a9d951563a service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:51.968912 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafbec0-5255-40f3-a258-07a9d951563a service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:51.969216 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafbec0-5255-40f3-a258-07a9d951563a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:51.977687 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafbec0-5255-40f3-a258-07a9d951563a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:51.979327 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafbec0-5255-40f3-a258-07a9d951563a service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:51.988480 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1338/2690] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:57:52.237938 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4524c-d333-4713-98d8-0f17f6152aa8 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:52.239616 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d4e4524c-d333-4713-98d8-0f17f6152aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:52.267797 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4524c-d333-4713-98d8-0f17f6152aa8 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:52.273648 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d346c-3207-4ba8-a525-75f763a136c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:52.274576 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d346c-3207-4ba8-a525-75f763a136c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:52.274860 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d346c-3207-4ba8-a525-75f763a136c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:52.275140 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d346c-3207-4ba8-a525-75f763a136c4 admin admin] PATH_INFO: `/v3/projects/f2c0aedd5cc44f13853822c762a9b1cc` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:52.276347 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d346c-3207-4ba8-a525-75f763a136c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2c0aedd5cc44f13853822c762a9b1cc)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:52.278454 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d346c-3207-4ba8-a525-75f763a136c4 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:52.410503 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:52.423162 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d86d346c-3207-4ba8-a525-75f763a136c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2c0aedd5cc44f13853822c762a9b1cc', 'request_id': 'req-d86d346c-3207-4ba8-a525-75f763a136c4'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:52.430083 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d86d346c-3207-4ba8-a525-75f763a136c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2c0aedd5cc44f13853822c762a9b1cc', 'request_id': 'req-d86d346c-3207-4ba8-a525-75f763a136c4'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:52.437198 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d86d346c-3207-4ba8-a525-75f763a136c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2c0aedd5cc44f13853822c762a9b1cc', 'request_id': 'req-d86d346c-3207-4ba8-a525-75f763a136c4'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:52.518296 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d86d346c-3207-4ba8-a525-75f763a136c4 admin admin] The token cache is being invalidate because project f2c0aedd5cc44f13853822c762a9b1cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:57:52.518648 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d86d346c-3207-4ba8-a525-75f763a136c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:52.519325 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d86d346c-3207-4ba8-a525-75f763a136c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:52.522513 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1353/2691] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:57:52 2022] DELETE /identity/v3/projects/f2c0aedd5cc44f13853822c762a9b1cc => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:52.527845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafbec0-5255-40f3-a258-07a9d951563a service nova] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:52.539973 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafbec0-5255-40f3-a258-07a9d951563a service nova] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:52.598989 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848d72c-8f7a-4eaa-8b49-206337404a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:52.599501 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848d72c-8f7a-4eaa-8b49-206337404a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:52.599868 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848d72c-8f7a-4eaa-8b49-206337404a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:52.600660 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848d72c-8f7a-4eaa-8b49-206337404a2a admin admin] PATH_INFO: `/v3/users/59152c9c8fcd417bbb0b77ada0416b8b` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:52.604117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848d72c-8f7a-4eaa-8b49-206337404a2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=59152c9c8fcd417bbb0b77ada0416b8b)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:52.604589 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848d72c-8f7a-4eaa-8b49-206337404a2a admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:52.774921 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-9848d72c-8f7a-4eaa-8b49-206337404a2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59152c9c8fcd417bbb0b77ada0416b8b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:52.788065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-9848d72c-8f7a-4eaa-8b49-206337404a2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59152c9c8fcd417bbb0b77ada0416b8b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:52.837478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-9848d72c-8f7a-4eaa-8b49-206337404a2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59152c9c8fcd417bbb0b77ada0416b8b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:52.838291 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-9848d72c-8f7a-4eaa-8b49-206337404a2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59152c9c8fcd417bbb0b77ada0416b8b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:52.838995 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-9848d72c-8f7a-4eaa-8b49-206337404a2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59152c9c8fcd417bbb0b77ada0416b8b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:52.853068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1339/2692] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:57:52 2022] DELETE /identity/v3/users/59152c9c8fcd417bbb0b77ada0416b8b => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:53.604627 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d346c-3207-4ba8-a525-75f763a136c4 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:53.606355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d86d346c-3207-4ba8-a525-75f763a136c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:53.635442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d346c-3207-4ba8-a525-75f763a136c4 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:53.728917 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09662f9e-1b36-4199-80d8-2067f6e672f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:53.730001 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09662f9e-1b36-4199-80d8-2067f6e672f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:53.730133 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09662f9e-1b36-4199-80d8-2067f6e672f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:53.730387 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09662f9e-1b36-4199-80d8-2067f6e672f6 admin admin] PATH_INFO: `/v3/projects/149c09fb51384cd89b62c1e1899b2f94` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:53.740065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09662f9e-1b36-4199-80d8-2067f6e672f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=149c09fb51384cd89b62c1e1899b2f94)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:53.740065 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09662f9e-1b36-4199-80d8-2067f6e672f6 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:53.896213 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-09662f9e-1b36-4199-80d8-2067f6e672f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '149c09fb51384cd89b62c1e1899b2f94', 'request_id': 'req-09662f9e-1b36-4199-80d8-2067f6e672f6'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:53.913152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-09662f9e-1b36-4199-80d8-2067f6e672f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '149c09fb51384cd89b62c1e1899b2f94', 'request_id': 'req-09662f9e-1b36-4199-80d8-2067f6e672f6'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:53.922676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-09662f9e-1b36-4199-80d8-2067f6e672f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '149c09fb51384cd89b62c1e1899b2f94', 'request_id': 'req-09662f9e-1b36-4199-80d8-2067f6e672f6'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:54.019603 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-09662f9e-1b36-4199-80d8-2067f6e672f6 admin admin] The token cache is being invalidate because project 149c09fb51384cd89b62c1e1899b2f94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:57:54.020018 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-09662f9e-1b36-4199-80d8-2067f6e672f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:54.020762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-09662f9e-1b36-4199-80d8-2067f6e672f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:57:54.027509 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1354/2693] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:57:53 2022] DELETE /identity/v3/projects/149c09fb51384cd89b62c1e1899b2f94 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:54.030134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0839736-4a5e-44c9-9d65-056cfa725ddb None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:54.030643 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0839736-4a5e-44c9-9d65-056cfa725ddb None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:54.030812 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0839736-4a5e-44c9-9d65-056cfa725ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:54.052099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-b0839736-4a5e-44c9-9d65-056cfa725ddb None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:54.053410 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-b0839736-4a5e-44c9-9d65-056cfa725ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:54.127345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:54.137555 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1340/2694] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:57:54.147517 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f6133-802e-465a-8269-1b232737ec80 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:54.147789 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f6133-802e-465a-8269-1b232737ec80 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:54.148017 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4f6133-802e-465a-8269-1b232737ec80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:54.167730 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-de4f6133-802e-465a-8269-1b232737ec80 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:54.168850 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-de4f6133-802e-465a-8269-1b232737ec80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:54.181278 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1355/2695] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:54.182520 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0839736-4a5e-44c9-9d65-056cfa725ddb None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:54.196518 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0839736-4a5e-44c9-9d65-056cfa725ddb None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:54.202289 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21466f1a-ded6-4e86-bd90-5583577343af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:54.202967 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21466f1a-ded6-4e86-bd90-5583577343af admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:54.203259 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21466f1a-ded6-4e86-bd90-5583577343af admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:54.203890 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21466f1a-ded6-4e86-bd90-5583577343af admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:54.204283 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21466f1a-ded6-4e86-bd90-5583577343af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:57:54.204594 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21466f1a-ded6-4e86-bd90-5583577343af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:54.206444 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21466f1a-ded6-4e86-bd90-5583577343af admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:54.220143 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1341/2696] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:57:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 19:57:54.224707 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f6133-802e-465a-8269-1b232737ec80 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:54.235512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4f6133-802e-465a-8269-1b232737ec80 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:54.241251 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57660be1-30fe-4496-8a54-aaa7bfdbd28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:54.242433 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57660be1-30fe-4496-8a54-aaa7bfdbd28b admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:54.242433 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57660be1-30fe-4496-8a54-aaa7bfdbd28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:54.242668 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57660be1-30fe-4496-8a54-aaa7bfdbd28b admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:54.244455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57660be1-30fe-4496-8a54-aaa7bfdbd28b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:54.246725 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57660be1-30fe-4496-8a54-aaa7bfdbd28b admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:54.343254 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1356/2697] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:57:54 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:54.347493 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21466f1a-ded6-4e86-bd90-5583577343af admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:54.366877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21466f1a-ded6-4e86-bd90-5583577343af admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:54.441194 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2523d928-b054-498a-80c0-59d3c71b1b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:54.442231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523d928-b054-498a-80c0-59d3c71b1b52 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:54.442517 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523d928-b054-498a-80c0-59d3c71b1b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:54.442793 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2523d928-b054-498a-80c0-59d3c71b1b52 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:54.443527 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2523d928-b054-498a-80c0-59d3c71b1b52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:54.445867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2523d928-b054-498a-80c0-59d3c71b1b52 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:54.563036 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1342/2698] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:57:54 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:54.568000 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57660be1-30fe-4496-8a54-aaa7bfdbd28b admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:54.578855 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57660be1-30fe-4496-8a54-aaa7bfdbd28b admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:54.583522 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebaa95-fa08-439a-9df9-fd592f644020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:54.584700 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebaa95-fa08-439a-9df9-fd592f644020 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:54.585011 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebaa95-fa08-439a-9df9-fd592f644020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:54.585290 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcebaa95-fa08-439a-9df9-fd592f644020 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:54.586143 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcebaa95-fa08-439a-9df9-fd592f644020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:54.587764 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcebaa95-fa08-439a-9df9-fd592f644020 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:54.598954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1357/2699] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:54.603048 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2523d928-b054-498a-80c0-59d3c71b1b52 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:54.613723 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2523d928-b054-498a-80c0-59d3c71b1b52 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:54.620134 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c01f8-3dd2-42f1-86cf-7f0b8dcc5348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:54.620830 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c01f8-3dd2-42f1-86cf-7f0b8dcc5348 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:54.626071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c01f8-3dd2-42f1-86cf-7f0b8dcc5348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:54.626071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c01f8-3dd2-42f1-86cf-7f0b8dcc5348 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:54.626071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c01f8-3dd2-42f1-86cf-7f0b8dcc5348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:54.626071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c01f8-3dd2-42f1-86cf-7f0b8dcc5348 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:54.649996 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1343/2700] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:54.650116 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebaa95-fa08-439a-9df9-fd592f644020 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:54.686466 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcebaa95-fa08-439a-9df9-fd592f644020 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:54.694876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9084c5-74a4-40d5-83fd-8c7b3aa8ffbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:54.696629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9084c5-74a4-40d5-83fd-8c7b3aa8ffbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:54.697377 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9084c5-74a4-40d5-83fd-8c7b3aa8ffbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:54.698018 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9084c5-74a4-40d5-83fd-8c7b3aa8ffbe admin admin] PATH_INFO: `/v3/projects/6e4635490934488e9749a80f7f2e64c7/users/6bcd17e63efa4fe299fd44df397680ef/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:54.717548 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9084c5-74a4-40d5-83fd-8c7b3aa8ffbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e4635490934488e9749a80f7f2e64c7, user_id=6bcd17e63efa4fe299fd44df397680ef, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:54.720506 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9084c5-74a4-40d5-83fd-8c7b3aa8ffbe admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:54.723511 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:54.796598 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1358/2701] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:54 2022] PUT /identity/v3/projects/6e4635490934488e9749a80f7f2e64c7/users/6bcd17e63efa4fe299fd44df397680ef/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:57:54.808745 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48371989-24e6-4068-bee2-1adf16c9e549 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:54.809194 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48371989-24e6-4068-bee2-1adf16c9e549 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:54.809569 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48371989-24e6-4068-bee2-1adf16c9e549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:54.845665 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-48371989-24e6-4068-bee2-1adf16c9e549 None None] MFA Rules not processed for user `6bcd17e63efa4fe299fd44df397680ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:54.850496 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-48371989-24e6-4068-bee2-1adf16c9e549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:54.850739 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed42255-36a6-42ff-ba98-52ac80531faf None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:54.850739 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed42255-36a6-42ff-ba98-52ac80531faf None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:54.850739 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed42255-36a6-42ff-ba98-52ac80531faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:54.872096 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-eed42255-36a6-42ff-ba98-52ac80531faf None None] MFA Rules not processed for user `bcdd3f30e42b41308b69e1233686cc17`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:54.873612 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-eed42255-36a6-42ff-ba98-52ac80531faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:54.931430 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1344/2702] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:54.936634 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48371989-24e6-4068-bee2-1adf16c9e549 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:54.941834 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1359/2703] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:55.005876 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:55.011147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48371989-24e6-4068-bee2-1adf16c9e549 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:55.015900 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021e052-bba0-442c-b52e-99be4f52673d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:55.016690 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021e052-bba0-442c-b52e-99be4f52673d admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:55.017260 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021e052-bba0-442c-b52e-99be4f52673d admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:55.017571 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9021e052-bba0-442c-b52e-99be4f52673d admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:55.018337 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9021e052-bba0-442c-b52e-99be4f52673d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:55.020378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9021e052-bba0-442c-b52e-99be4f52673d admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:55.143487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1345/2704] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:57:54 2022] POST /identity/v3/projects => generated 385 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:55.148256 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed42255-36a6-42ff-ba98-52ac80531faf None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:55.159472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed42255-36a6-42ff-ba98-52ac80531faf None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:55.229112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f3edee-3418-4c78-92e2-7d2b99102f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:55.229819 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3edee-3418-4c78-92e2-7d2b99102f55 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:55.230109 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3edee-3418-4c78-92e2-7d2b99102f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:55.233273 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3edee-3418-4c78-92e2-7d2b99102f55 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:55.233273 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f3edee-3418-4c78-92e2-7d2b99102f55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:55.233273 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f3edee-3418-4c78-92e2-7d2b99102f55 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:55.344177 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1360/2705] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:57:55 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:55.345087 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021e052-bba0-442c-b52e-99be4f52673d admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:55.356172 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9021e052-bba0-442c-b52e-99be4f52673d admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:55.361755 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5ed1f-f4a6-4f04-8e00-05b6a1ea0d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:55.362745 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5ed1f-f4a6-4f04-8e00-05b6a1ea0d6a admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:55.363533 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5ed1f-f4a6-4f04-8e00-05b6a1ea0d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:55.363979 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f5ed1f-f4a6-4f04-8e00-05b6a1ea0d6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:55.365019 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5ed1f-f4a6-4f04-8e00-05b6a1ea0d6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:55.367281 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f5ed1f-f4a6-4f04-8e00-05b6a1ea0d6a admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:55.382065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1346/2706] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:55.383537 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f3edee-3418-4c78-92e2-7d2b99102f55 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:55.400087 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f3edee-3418-4c78-92e2-7d2b99102f55 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:55.400789 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee7367b-31fb-432b-8cfa-845ad4fbd2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:55.401756 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7367b-31fb-432b-8cfa-845ad4fbd2f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:55.402356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7367b-31fb-432b-8cfa-845ad4fbd2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:55.402707 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee7367b-31fb-432b-8cfa-845ad4fbd2f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:55.403520 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee7367b-31fb-432b-8cfa-845ad4fbd2f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:55.405464 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee7367b-31fb-432b-8cfa-845ad4fbd2f2 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:55.420542 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1361/2707] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:55.424047 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5ed1f-f4a6-4f04-8e00-05b6a1ea0d6a admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:55.436108 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f5ed1f-f4a6-4f04-8e00-05b6a1ea0d6a admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:55.441405 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1ccb9-58ac-487c-966d-95bc950f6874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:55.442843 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1ccb9-58ac-487c-966d-95bc950f6874 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:55.443139 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1ccb9-58ac-487c-966d-95bc950f6874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:55.444222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1ccb9-58ac-487c-966d-95bc950f6874 admin admin] PATH_INFO: `/v3/projects/151c776d47cf4b769fad976451d30ed7/users/4f78702ba1804288a4a395cfdf3f62b0/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:55.463792 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1ccb9-58ac-487c-966d-95bc950f6874 admin admin] RBAC: Authorizing `identity:create_grant(project_id=151c776d47cf4b769fad976451d30ed7, user_id=4f78702ba1804288a4a395cfdf3f62b0, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:55.466688 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1ccb9-58ac-487c-966d-95bc950f6874 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:55.569381 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1347/2708] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:55 2022] PUT /identity/v3/projects/151c776d47cf4b769fad976451d30ed7/users/4f78702ba1804288a4a395cfdf3f62b0/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:55.580474 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e3b71-253b-4c78-b3a1-feb5659c34db None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:55.580878 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e3b71-253b-4c78-b3a1-feb5659c34db None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:55.581262 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e3b71-253b-4c78-b3a1-feb5659c34db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:55.610123 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-110e3b71-253b-4c78-b3a1-feb5659c34db None None] MFA Rules not processed for user `4f78702ba1804288a4a395cfdf3f62b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:55.611391 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-110e3b71-253b-4c78-b3a1-feb5659c34db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:55.691910 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1362/2709] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 19:57:55.696199 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1ccb9-58ac-487c-966d-95bc950f6874 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:55.771688 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1ccb9-58ac-487c-966d-95bc950f6874 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:55.784219 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee2caf-02e4-4492-8fee-d3f06d906ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:55.785138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee2caf-02e4-4492-8fee-d3f06d906ed4 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:55.785531 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee2caf-02e4-4492-8fee-d3f06d906ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:55.785903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fee2caf-02e4-4492-8fee-d3f06d906ed4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:55.786786 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee2caf-02e4-4492-8fee-d3f06d906ed4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:55.789583 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fee2caf-02e4-4492-8fee-d3f06d906ed4 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:55.877205 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1348/2710] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:57:55 2022] POST /identity/v3/projects => generated 385 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:57:55.881512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e3b71-253b-4c78-b3a1-feb5659c34db None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:55.892783 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e3b71-253b-4c78-b3a1-feb5659c34db None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:55.898697 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:55.952370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb9b0a-0a35-402b-bd5e-5abaa023e853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:55.953179 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb9b0a-0a35-402b-bd5e-5abaa023e853 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:55.955876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb9b0a-0a35-402b-bd5e-5abaa023e853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:55.955876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb9b0a-0a35-402b-bd5e-5abaa023e853 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:55.955876 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb9b0a-0a35-402b-bd5e-5abaa023e853 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:55.956341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84eb9b0a-0a35-402b-bd5e-5abaa023e853 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:56.059320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1363/2711] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:57:55 2022] POST /identity/v3/users => generated 299 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:57:56.063317 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee2caf-02e4-4492-8fee-d3f06d906ed4 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:56.074603 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fee2caf-02e4-4492-8fee-d3f06d906ed4 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:56.079261 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bad23-be5b-40a2-a947-ad2e2aa2e6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:56.080073 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bad23-be5b-40a2-a947-ad2e2aa2e6dd admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:56.080356 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bad23-be5b-40a2-a947-ad2e2aa2e6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:56.080631 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4bad23-be5b-40a2-a947-ad2e2aa2e6dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:56.081441 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4bad23-be5b-40a2-a947-ad2e2aa2e6dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:56.083036 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4bad23-be5b-40a2-a947-ad2e2aa2e6dd admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:56.094715 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1349/2712] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:56.098912 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb9b0a-0a35-402b-bd5e-5abaa023e853 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:56.109681 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84eb9b0a-0a35-402b-bd5e-5abaa023e853 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:56.114306 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e99cccb-de67-4784-acb9-96c5018df05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:56.115187 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99cccb-de67-4784-acb9-96c5018df05b admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:56.115480 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99cccb-de67-4784-acb9-96c5018df05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:56.115769 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99cccb-de67-4784-acb9-96c5018df05b admin admin] PATH_INFO: `/v3/projects/0cbbd5075db0429aa16ba852a35fe116/users/bbff8b2b93a0439d877df21b9bbe73d6/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:56.130487 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e99cccb-de67-4784-acb9-96c5018df05b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cbbd5075db0429aa16ba852a35fe116, user_id=bbff8b2b93a0439d877df21b9bbe73d6, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:56.133358 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e99cccb-de67-4784-acb9-96c5018df05b admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:56.212798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1364/2713] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:56 2022] PUT /identity/v3/projects/0cbbd5075db0429aa16ba852a35fe116/users/bbff8b2b93a0439d877df21b9bbe73d6/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:56.213015 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bad23-be5b-40a2-a947-ad2e2aa2e6dd admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:56.222168 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4bad23-be5b-40a2-a947-ad2e2aa2e6dd admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:56.281417 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74f4bcb-4989-43e5-8e9c-74ceea6a3916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:56.282186 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f4bcb-4989-43e5-8e9c-74ceea6a3916 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:56.282380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f4bcb-4989-43e5-8e9c-74ceea6a3916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:56.282657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74f4bcb-4989-43e5-8e9c-74ceea6a3916 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:56.283380 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f4bcb-4989-43e5-8e9c-74ceea6a3916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:56.284989 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74f4bcb-4989-43e5-8e9c-74ceea6a3916 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:56.295882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1350/2714] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:57:56 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:57:56.300029 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e99cccb-de67-4784-acb9-96c5018df05b admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:56.311227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e99cccb-de67-4784-acb9-96c5018df05b admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:56.317636 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32801525-c585-4696-a9b7-7d165834f63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:56.317636 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32801525-c585-4696-a9b7-7d165834f63e admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:56.317636 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32801525-c585-4696-a9b7-7d165834f63e admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:56.317636 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32801525-c585-4696-a9b7-7d165834f63e admin admin] PATH_INFO: `/v3/projects/0cbbd5075db0429aa16ba852a35fe116/users/bbff8b2b93a0439d877df21b9bbe73d6/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:56.319891 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32801525-c585-4696-a9b7-7d165834f63e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cbbd5075db0429aa16ba852a35fe116, user_id=bbff8b2b93a0439d877df21b9bbe73d6, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:56.323124 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32801525-c585-4696-a9b7-7d165834f63e admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:56.336331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1365/2715] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:57:56 2022] PUT /identity/v3/projects/0cbbd5075db0429aa16ba852a35fe116/users/bbff8b2b93a0439d877df21b9bbe73d6/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:57:56.403847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15412717-8627-4ece-9e1a-c002185ab813 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:56.404138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15412717-8627-4ece-9e1a-c002185ab813 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:56.404422 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15412717-8627-4ece-9e1a-c002185ab813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:56.434502 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-15412717-8627-4ece-9e1a-c002185ab813 None None] MFA Rules not processed for user `bbff8b2b93a0439d877df21b9bbe73d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:57:56.435808 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-15412717-8627-4ece-9e1a-c002185ab813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:56.440841 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:57:56.512265 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1351/2716] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:57:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:57:56.524917 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32801525-c585-4696-a9b7-7d165834f63e admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:56.526776 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-32801525-c585-4696-a9b7-7d165834f63e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:56.585998 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32801525-c585-4696-a9b7-7d165834f63e admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:56.589710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bf666-961c-4f75-a114-180bc22c44b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:56.590386 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bf666-961c-4f75-a114-180bc22c44b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:56.590668 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bf666-961c-4f75-a114-180bc22c44b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:56.590944 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679bf666-961c-4f75-a114-180bc22c44b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:56.647003 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679bf666-961c-4f75-a114-180bc22c44b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:56.648630 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679bf666-961c-4f75-a114-180bc22c44b8 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:56.657568 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1366/2717] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:57:58.659698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15412717-8627-4ece-9e1a-c002185ab813 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:57:58.661209 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-15412717-8627-4ece-9e1a-c002185ab813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:57:58.674138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15412717-8627-4ece-9e1a-c002185ab813 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:57:58.678376 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234ced2-33ce-4f96-9c66-063cb1f84813 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:57:58.679061 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234ced2-33ce-4f96-9c66-063cb1f84813 service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:57:58.679928 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234ced2-33ce-4f96-9c66-063cb1f84813 service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:57:58.679928 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6234ced2-33ce-4f96-9c66-063cb1f84813 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:57:58.758394 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6234ced2-33ce-4f96-9c66-063cb1f84813 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:57:58.759908 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6234ced2-33ce-4f96-9c66-063cb1f84813 service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:57:58.770839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1352/2718] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:57:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:58:00.342384 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bf666-961c-4f75-a114-180bc22c44b8 service cinder] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:00.351846 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679bf666-961c-4f75-a114-180bc22c44b8 service cinder] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:00.355678 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf5c5a-5e6a-4cc4-8df7-fc2064d3c5f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:00.356399 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf5c5a-5e6a-4cc4-8df7-fc2064d3c5f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:00.356758 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf5c5a-5e6a-4cc4-8df7-fc2064d3c5f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:00.357149 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf5c5a-5e6a-4cc4-8df7-fc2064d3c5f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:00.366586 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf5c5a-5e6a-4cc4-8df7-fc2064d3c5f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:00.368113 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf5c5a-5e6a-4cc4-8df7-fc2064d3c5f9 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:00.373362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:58:00.378190 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1367/2719] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:58:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:58:04.978219 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234ced2-33ce-4f96-9c66-063cb1f84813 service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:05.041977 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6234ced2-33ce-4f96-9c66-063cb1f84813 service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:05.046625 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131140f-a649-4eec-b9fe-9548f2a9a400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:05.047968 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131140f-a649-4eec-b9fe-9548f2a9a400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:05.048257 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131140f-a649-4eec-b9fe-9548f2a9a400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:05.048549 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131140f-a649-4eec-b9fe-9548f2a9a400 admin admin] PATH_INFO: `/v3/users/6bcd17e63efa4fe299fd44df397680ef` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:05.049756 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131140f-a649-4eec-b9fe-9548f2a9a400 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bcd17e63efa4fe299fd44df397680ef)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:05.051809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d131140f-a649-4eec-b9fe-9548f2a9a400 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:05.244740 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:58:05.249678 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d131140f-a649-4eec-b9fe-9548f2a9a400 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bcd17e63efa4fe299fd44df397680ef'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:05.261848 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d131140f-a649-4eec-b9fe-9548f2a9a400 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bcd17e63efa4fe299fd44df397680ef'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:05.321118 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d131140f-a649-4eec-b9fe-9548f2a9a400 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bcd17e63efa4fe299fd44df397680ef'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:05.321749 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d131140f-a649-4eec-b9fe-9548f2a9a400 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bcd17e63efa4fe299fd44df397680ef'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:05.322358 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d131140f-a649-4eec-b9fe-9548f2a9a400 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bcd17e63efa4fe299fd44df397680ef'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:05.335573 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1353/2720] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:58:04 2022] DELETE /identity/v3/users/6bcd17e63efa4fe299fd44df397680ef => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:58:05.340080 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec147b-3460-4249-9868-6b0c71f101e6 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:05.340557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec147b-3460-4249-9868-6b0c71f101e6 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:05.340679 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec147b-3460-4249-9868-6b0c71f101e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:05.358276 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-c4ec147b-3460-4249-9868-6b0c71f101e6 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:58:05.359463 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-c4ec147b-3460-4249-9868-6b0c71f101e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:58:05.417924 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1368/2721] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:58:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:58:05.428617 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131140f-a649-4eec-b9fe-9548f2a9a400 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:05.430163 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d131140f-a649-4eec-b9fe-9548f2a9a400 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:58:05.485721 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d131140f-a649-4eec-b9fe-9548f2a9a400 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:05.490055 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b349bf-be76-43d9-b0e8-4e64ccbbf074 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:05.490707 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b349bf-be76-43d9-b0e8-4e64ccbbf074 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:05.490995 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b349bf-be76-43d9-b0e8-4e64ccbbf074 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:05.491277 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b349bf-be76-43d9-b0e8-4e64ccbbf074 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:05.500275 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b349bf-be76-43d9-b0e8-4e64ccbbf074 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:05.501747 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b349bf-be76-43d9-b0e8-4e64ccbbf074 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:05.511471 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1354/2722] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:58:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:58:05.944231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec147b-3460-4249-9868-6b0c71f101e6 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:05.945794 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-c4ec147b-3460-4249-9868-6b0c71f101e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:58:05.961668 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec147b-3460-4249-9868-6b0c71f101e6 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:05.967179 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c1b14-1e7a-48b4-b2fc-302ad2236160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:05.968031 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c1b14-1e7a-48b4-b2fc-302ad2236160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:05.968317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c1b14-1e7a-48b4-b2fc-302ad2236160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:05.968599 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7c1b14-1e7a-48b4-b2fc-302ad2236160 admin admin] PATH_INFO: `/v3/projects/6e4635490934488e9749a80f7f2e64c7` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:05.969887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7c1b14-1e7a-48b4-b2fc-302ad2236160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e4635490934488e9749a80f7f2e64c7)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:05.971880 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7c1b14-1e7a-48b4-b2fc-302ad2236160 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:06.100555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-ee7c1b14-1e7a-48b4-b2fc-302ad2236160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e4635490934488e9749a80f7f2e64c7', 'request_id': 'req-ee7c1b14-1e7a-48b4-b2fc-302ad2236160'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:06.106787 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-ee7c1b14-1e7a-48b4-b2fc-302ad2236160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e4635490934488e9749a80f7f2e64c7', 'request_id': 'req-ee7c1b14-1e7a-48b4-b2fc-302ad2236160'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:06.113200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-ee7c1b14-1e7a-48b4-b2fc-302ad2236160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e4635490934488e9749a80f7f2e64c7', 'request_id': 'req-ee7c1b14-1e7a-48b4-b2fc-302ad2236160'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:06.213348 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-ee7c1b14-1e7a-48b4-b2fc-302ad2236160 admin admin] The token cache is being invalidate because project 6e4635490934488e9749a80f7f2e64c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:58:06.213641 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-ee7c1b14-1e7a-48b4-b2fc-302ad2236160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:06.214267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-ee7c1b14-1e7a-48b4-b2fc-302ad2236160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:06.215828 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1369/2723] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:58:05 2022] DELETE /identity/v3/projects/6e4635490934488e9749a80f7f2e64c7 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:58:06.219847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b349bf-be76-43d9-b0e8-4e64ccbbf074 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:06.229614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b349bf-be76-43d9-b0e8-4e64ccbbf074 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:06.285847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e756de-e9cf-45ba-9a2c-0e97eee90bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:06.287499 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e756de-e9cf-45ba-9a2c-0e97eee90bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:06.287854 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e756de-e9cf-45ba-9a2c-0e97eee90bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:06.288140 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e756de-e9cf-45ba-9a2c-0e97eee90bf6 admin admin] PATH_INFO: `/v3/users/4f78702ba1804288a4a395cfdf3f62b0` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:06.289399 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e756de-e9cf-45ba-9a2c-0e97eee90bf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f78702ba1804288a4a395cfdf3f62b0)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:06.291503 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e756de-e9cf-45ba-9a2c-0e97eee90bf6 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:06.471960 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c5e756de-e9cf-45ba-9a2c-0e97eee90bf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f78702ba1804288a4a395cfdf3f62b0'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:06.483424 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:58:06.484636 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c5e756de-e9cf-45ba-9a2c-0e97eee90bf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f78702ba1804288a4a395cfdf3f62b0'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:06.521799 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c5e756de-e9cf-45ba-9a2c-0e97eee90bf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f78702ba1804288a4a395cfdf3f62b0'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:06.522439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c5e756de-e9cf-45ba-9a2c-0e97eee90bf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f78702ba1804288a4a395cfdf3f62b0'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:06.523070 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-c5e756de-e9cf-45ba-9a2c-0e97eee90bf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f78702ba1804288a4a395cfdf3f62b0'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:06.537755 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1355/2724] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:58:06 2022] DELETE /identity/v3/users/4f78702ba1804288a4a395cfdf3f62b0 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:58:06.963788 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c1b14-1e7a-48b4-b2fc-302ad2236160 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:06.965613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ee7c1b14-1e7a-48b4-b2fc-302ad2236160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:58:06.982955 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7c1b14-1e7a-48b4-b2fc-302ad2236160 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:07.044357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea99210-c7ea-4a8d-8af5-17970048517c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:07.045257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea99210-c7ea-4a8d-8af5-17970048517c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:07.045542 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea99210-c7ea-4a8d-8af5-17970048517c admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:07.045818 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea99210-c7ea-4a8d-8af5-17970048517c admin admin] PATH_INFO: `/v3/projects/151c776d47cf4b769fad976451d30ed7` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:07.047647 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea99210-c7ea-4a8d-8af5-17970048517c admin admin] RBAC: Authorizing `identity:delete_project(project_id=151c776d47cf4b769fad976451d30ed7)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:07.049616 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea99210-c7ea-4a8d-8af5-17970048517c admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:07.122093 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:58:07.138981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-1ea99210-c7ea-4a8d-8af5-17970048517c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '151c776d47cf4b769fad976451d30ed7', 'request_id': 'req-1ea99210-c7ea-4a8d-8af5-17970048517c'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:07.144487 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-1ea99210-c7ea-4a8d-8af5-17970048517c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '151c776d47cf4b769fad976451d30ed7', 'request_id': 'req-1ea99210-c7ea-4a8d-8af5-17970048517c'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:07.151716 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-1ea99210-c7ea-4a8d-8af5-17970048517c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '151c776d47cf4b769fad976451d30ed7', 'request_id': 'req-1ea99210-c7ea-4a8d-8af5-17970048517c'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:07.196704 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-1ea99210-c7ea-4a8d-8af5-17970048517c admin admin] The token cache is being invalidate because project 151c776d47cf4b769fad976451d30ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:58:07.197036 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-1ea99210-c7ea-4a8d-8af5-17970048517c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:07.197723 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-1ea99210-c7ea-4a8d-8af5-17970048517c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:07.199655 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1370/2725] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:58:06 2022] DELETE /identity/v3/projects/151c776d47cf4b769fad976451d30ed7 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:58:07.203670 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e756de-e9cf-45ba-9a2c-0e97eee90bf6 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:07.205409 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-c5e756de-e9cf-45ba-9a2c-0e97eee90bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:58:07.220386 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e756de-e9cf-45ba-9a2c-0e97eee90bf6 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:07.285222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599a17c-0b52-4632-be6c-ad7bf446885b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:07.286290 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599a17c-0b52-4632-be6c-ad7bf446885b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:07.286426 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599a17c-0b52-4632-be6c-ad7bf446885b admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:07.286655 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599a17c-0b52-4632-be6c-ad7bf446885b admin admin] PATH_INFO: `/v3/users/bbff8b2b93a0439d877df21b9bbe73d6` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:07.287861 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599a17c-0b52-4632-be6c-ad7bf446885b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbff8b2b93a0439d877df21b9bbe73d6)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:07.289816 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599a17c-0b52-4632-be6c-ad7bf446885b admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:07.432267 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-3599a17c-0b52-4632-be6c-ad7bf446885b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbff8b2b93a0439d877df21b9bbe73d6'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:07.444346 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-3599a17c-0b52-4632-be6c-ad7bf446885b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbff8b2b93a0439d877df21b9bbe73d6'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:07.480488 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-3599a17c-0b52-4632-be6c-ad7bf446885b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbff8b2b93a0439d877df21b9bbe73d6'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:07.481122 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-3599a17c-0b52-4632-be6c-ad7bf446885b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbff8b2b93a0439d877df21b9bbe73d6'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:07.481716 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-3599a17c-0b52-4632-be6c-ad7bf446885b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbff8b2b93a0439d877df21b9bbe73d6'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:07.494610 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1356/2726] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:58:07 2022] DELETE /identity/v3/users/bbff8b2b93a0439d877df21b9bbe73d6 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:58:07.887836 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea99210-c7ea-4a8d-8af5-17970048517c admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:07.889628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-1ea99210-c7ea-4a8d-8af5-17970048517c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:58:07.904446 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea99210-c7ea-4a8d-8af5-17970048517c admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:07.963472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221b04c-b567-4901-b23c-8ffbdeb84d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:07.964341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221b04c-b567-4901-b23c-8ffbdeb84d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:07.964922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221b04c-b567-4901-b23c-8ffbdeb84d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:07.965055 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221b04c-b567-4901-b23c-8ffbdeb84d6a admin admin] PATH_INFO: `/v3/projects/0cbbd5075db0429aa16ba852a35fe116` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:07.966343 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221b04c-b567-4901-b23c-8ffbdeb84d6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cbbd5075db0429aa16ba852a35fe116)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:07.971874 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221b04c-b567-4901-b23c-8ffbdeb84d6a admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:08.126305 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-0221b04c-b567-4901-b23c-8ffbdeb84d6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cbbd5075db0429aa16ba852a35fe116', 'request_id': 'req-0221b04c-b567-4901-b23c-8ffbdeb84d6a'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:08.132160 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-0221b04c-b567-4901-b23c-8ffbdeb84d6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cbbd5075db0429aa16ba852a35fe116', 'request_id': 'req-0221b04c-b567-4901-b23c-8ffbdeb84d6a'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:08.138409 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-0221b04c-b567-4901-b23c-8ffbdeb84d6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cbbd5075db0429aa16ba852a35fe116', 'request_id': 'req-0221b04c-b567-4901-b23c-8ffbdeb84d6a'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:08.199915 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-0221b04c-b567-4901-b23c-8ffbdeb84d6a admin admin] The token cache is being invalidate because project 0cbbd5075db0429aa16ba852a35fe116 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:58:08.200202 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-0221b04c-b567-4901-b23c-8ffbdeb84d6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:08.200821 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-0221b04c-b567-4901-b23c-8ffbdeb84d6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:08.202440 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1371/2727] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:58:07 2022] DELETE /identity/v3/projects/0cbbd5075db0429aa16ba852a35fe116 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:58:08.207575 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2c2da-51db-4b6d-88be-2ee75f47cfa1 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:08.207872 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2c2da-51db-4b6d-88be-2ee75f47cfa1 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:08.209125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2c2da-51db-4b6d-88be-2ee75f47cfa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:08.227086 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-62d2c2da-51db-4b6d-88be-2ee75f47cfa1 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:58:08.228271 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-62d2c2da-51db-4b6d-88be-2ee75f47cfa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:58:08.277768 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:58:08.285845 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1357/2728] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:58:08.302104 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b3f82-a2f0-416e-bdeb-38193435da4f None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:08.302480 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b3f82-a2f0-416e-bdeb-38193435da4f None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:08.302774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b3f82-a2f0-416e-bdeb-38193435da4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:08.321645 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-c90b3f82-a2f0-416e-bdeb-38193435da4f None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:58:08.322873 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-c90b3f82-a2f0-416e-bdeb-38193435da4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:58:08.331014 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1372/2729] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:58:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:58:08.335987 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2c2da-51db-4b6d-88be-2ee75f47cfa1 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:08.345651 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2c2da-51db-4b6d-88be-2ee75f47cfa1 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:08.351005 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc154b7-b7c9-4d3f-a026-defa77778b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:08.351687 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc154b7-b7c9-4d3f-a026-defa77778b3d admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:08.351974 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc154b7-b7c9-4d3f-a026-defa77778b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:08.352252 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc154b7-b7c9-4d3f-a026-defa77778b3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:08.353012 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc154b7-b7c9-4d3f-a026-defa77778b3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:58:08.353327 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc154b7-b7c9-4d3f-a026-defa77778b3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:08.354925 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc154b7-b7c9-4d3f-a026-defa77778b3d admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:08.366902 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1358/2730] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:58:08.370804 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b3f82-a2f0-416e-bdeb-38193435da4f None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:08.380790 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b3f82-a2f0-416e-bdeb-38193435da4f None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:08.385842 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3900ef-26dc-4b68-a01e-855224e73845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:08.386510 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3900ef-26dc-4b68-a01e-855224e73845 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:08.386796 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3900ef-26dc-4b68-a01e-855224e73845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:08.387145 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3900ef-26dc-4b68-a01e-855224e73845 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:08.387826 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3900ef-26dc-4b68-a01e-855224e73845 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:08.389826 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3900ef-26dc-4b68-a01e-855224e73845 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:08.474059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1373/2731] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:58:08 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:58:08.477146 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc154b7-b7c9-4d3f-a026-defa77778b3d admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:08.485071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc154b7-b7c9-4d3f-a026-defa77778b3d admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:08.549681 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030bcdf-aa51-46d7-ba17-2346cdc4408e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:08.550362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030bcdf-aa51-46d7-ba17-2346cdc4408e admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:08.550645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030bcdf-aa51-46d7-ba17-2346cdc4408e admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:08.550923 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0030bcdf-aa51-46d7-ba17-2346cdc4408e admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:08.551617 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0030bcdf-aa51-46d7-ba17-2346cdc4408e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:08.553740 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0030bcdf-aa51-46d7-ba17-2346cdc4408e admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:08.636538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1359/2732] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:58:08 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:58:08.640289 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3900ef-26dc-4b68-a01e-855224e73845 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:08.651460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3900ef-26dc-4b68-a01e-855224e73845 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:08.656214 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ff26e-0828-48ec-8d86-f7ec79c06be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:08.657033 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ff26e-0828-48ec-8d86-f7ec79c06be0 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:08.657272 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ff26e-0828-48ec-8d86-f7ec79c06be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:08.657562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8ff26e-0828-48ec-8d86-f7ec79c06be0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:08.658267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ff26e-0828-48ec-8d86-f7ec79c06be0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:08.659971 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8ff26e-0828-48ec-8d86-f7ec79c06be0 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:08.671744 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1374/2733] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:58:08.675782 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030bcdf-aa51-46d7-ba17-2346cdc4408e admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:08.686022 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0030bcdf-aa51-46d7-ba17-2346cdc4408e admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:08.690914 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad714ca-0b46-4820-bcbf-794d0a951b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:08.691587 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad714ca-0b46-4820-bcbf-794d0a951b3f admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:08.691877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad714ca-0b46-4820-bcbf-794d0a951b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:08.692157 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad714ca-0b46-4820-bcbf-794d0a951b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:08.693425 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad714ca-0b46-4820-bcbf-794d0a951b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:08.695058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad714ca-0b46-4820-bcbf-794d0a951b3f admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:08.705150 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1360/2734] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:58:08.710724 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ff26e-0828-48ec-8d86-f7ec79c06be0 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:08.717475 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:58:08.721320 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8ff26e-0828-48ec-8d86-f7ec79c06be0 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:08.725998 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2f27c-40bc-4070-8227-c916acc51a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:08.726885 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2f27c-40bc-4070-8227-c916acc51a95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:08.727169 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2f27c-40bc-4070-8227-c916acc51a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:08.727443 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f2f27c-40bc-4070-8227-c916acc51a95 admin admin] PATH_INFO: `/v3/projects/496447d6c65b4666ae621c200a04dc65/users/2e4d2d7ac86843eb9ee63273204e6648/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:08.742499 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2f27c-40bc-4070-8227-c916acc51a95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=496447d6c65b4666ae621c200a04dc65, user_id=2e4d2d7ac86843eb9ee63273204e6648, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:08.745418 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f2f27c-40bc-4070-8227-c916acc51a95 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:08.812890 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1375/2735] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:58:08 2022] PUT /identity/v3/projects/496447d6c65b4666ae621c200a04dc65/users/2e4d2d7ac86843eb9ee63273204e6648/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:58:08.825020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a327eb7-1f9a-46ef-85b6-804bc4bf3043 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:08.825340 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a327eb7-1f9a-46ef-85b6-804bc4bf3043 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:08.825625 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a327eb7-1f9a-46ef-85b6-804bc4bf3043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:08.853234 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-4a327eb7-1f9a-46ef-85b6-804bc4bf3043 None None] MFA Rules not processed for user `2e4d2d7ac86843eb9ee63273204e6648`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:58:08.854479 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-4a327eb7-1f9a-46ef-85b6-804bc4bf3043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:58:08.917416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1361/2736] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:58:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:58:08.929917 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2f27c-40bc-4070-8227-c916acc51a95 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:08.930922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-59f2f27c-40bc-4070-8227-c916acc51a95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:58:08.987646 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f2f27c-40bc-4070-8227-c916acc51a95 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:08.991914 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1bb9b6-43b4-4ad1-9356-4912610f3fbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:08.992559 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bb9b6-43b4-4ad1-9356-4912610f3fbb service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:08.992851 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bb9b6-43b4-4ad1-9356-4912610f3fbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:08.993171 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1bb9b6-43b4-4ad1-9356-4912610f3fbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:09.001515 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1bb9b6-43b4-4ad1-9356-4912610f3fbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:09.002958 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1bb9b6-43b4-4ad1-9356-4912610f3fbb service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:09.012440 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1376/2737] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:58:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 19:58:12.050136 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a327eb7-1f9a-46ef-85b6-804bc4bf3043 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:12.111113 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:58:12.116665 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a327eb7-1f9a-46ef-85b6-804bc4bf3043 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:12.121317 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bca59-5de0-4960-ae57-eee69a61626f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:12.122193 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bca59-5de0-4960-ae57-eee69a61626f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:12.122482 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bca59-5de0-4960-ae57-eee69a61626f admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:12.122766 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bca59-5de0-4960-ae57-eee69a61626f admin admin] PATH_INFO: `/v3/users/2e4d2d7ac86843eb9ee63273204e6648` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:12.124051 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4bca59-5de0-4960-ae57-eee69a61626f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e4d2d7ac86843eb9ee63273204e6648)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:12.126021 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4bca59-5de0-4960-ae57-eee69a61626f admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:12.299649 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-db4bca59-5de0-4960-ae57-eee69a61626f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e4d2d7ac86843eb9ee63273204e6648'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:12.312292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-db4bca59-5de0-4960-ae57-eee69a61626f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e4d2d7ac86843eb9ee63273204e6648'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:12.389093 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-db4bca59-5de0-4960-ae57-eee69a61626f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e4d2d7ac86843eb9ee63273204e6648'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:12.389856 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-db4bca59-5de0-4960-ae57-eee69a61626f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e4d2d7ac86843eb9ee63273204e6648'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:12.390559 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-db4bca59-5de0-4960-ae57-eee69a61626f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e4d2d7ac86843eb9ee63273204e6648'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:12.403745 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1362/2738] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:58:12 2022] DELETE /identity/v3/users/2e4d2d7ac86843eb9ee63273204e6648 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 19:58:12.408215 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6af75-bce8-4a33-8e16-fc56d990fdd1 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:12.408676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6af75-bce8-4a33-8e16-fc56d990fdd1 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:12.408910 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6af75-bce8-4a33-8e16-fc56d990fdd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:12.426393 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-10f6af75-bce8-4a33-8e16-fc56d990fdd1 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:58:12.427591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-10f6af75-bce8-4a33-8e16-fc56d990fdd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:58:12.485358 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1377/2739] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:58:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:58:12.495382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bca59-5de0-4960-ae57-eee69a61626f admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:12.496776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-db4bca59-5de0-4960-ae57-eee69a61626f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:58:12.554370 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bca59-5de0-4960-ae57-eee69a61626f admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:12.557945 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb542e37-ffb7-4e88-95a7-dd1e11de2353 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:12.558588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb542e37-ffb7-4e88-95a7-dd1e11de2353 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:12.558874 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb542e37-ffb7-4e88-95a7-dd1e11de2353 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:12.559152 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb542e37-ffb7-4e88-95a7-dd1e11de2353 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:12.568102 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb542e37-ffb7-4e88-95a7-dd1e11de2353 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:12.569698 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb542e37-ffb7-4e88-95a7-dd1e11de2353 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:12.579307 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1363/2740] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:58:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:58:12.990913 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6af75-bce8-4a33-8e16-fc56d990fdd1 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:12.992339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-10f6af75-bce8-4a33-8e16-fc56d990fdd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:58:13.004764 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6af75-bce8-4a33-8e16-fc56d990fdd1 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:13.010113 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ce457-fe80-451c-8756-54242284a556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:13.010990 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ce457-fe80-451c-8756-54242284a556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:13.011278 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ce457-fe80-451c-8756-54242284a556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:13.011564 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ce457-fe80-451c-8756-54242284a556 admin admin] PATH_INFO: `/v3/projects/496447d6c65b4666ae621c200a04dc65` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:13.012764 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ce457-fe80-451c-8756-54242284a556 admin admin] RBAC: Authorizing `identity:delete_project(project_id=496447d6c65b4666ae621c200a04dc65)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:13.014719 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ce457-fe80-451c-8756-54242284a556 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:13.143176 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d29ce457-fe80-451c-8756-54242284a556 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '496447d6c65b4666ae621c200a04dc65', 'request_id': 'req-d29ce457-fe80-451c-8756-54242284a556'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:13.148723 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d29ce457-fe80-451c-8756-54242284a556 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '496447d6c65b4666ae621c200a04dc65', 'request_id': 'req-d29ce457-fe80-451c-8756-54242284a556'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:13.154889 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d29ce457-fe80-451c-8756-54242284a556 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '496447d6c65b4666ae621c200a04dc65', 'request_id': 'req-d29ce457-fe80-451c-8756-54242284a556'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:13.250553 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d29ce457-fe80-451c-8756-54242284a556 admin admin] The token cache is being invalidate because project 496447d6c65b4666ae621c200a04dc65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:58:13.250838 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d29ce457-fe80-451c-8756-54242284a556 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:13.251450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d29ce457-fe80-451c-8756-54242284a556 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:13.253049 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1378/2741] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:58:12 2022] DELETE /identity/v3/projects/496447d6c65b4666ae621c200a04dc65 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:58:13.258055 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4209b-815a-459f-b32b-19282279abda None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:13.258563 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4209b-815a-459f-b32b-19282279abda None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:13.258667 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4209b-815a-459f-b32b-19282279abda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:13.276683 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-c6b4209b-815a-459f-b32b-19282279abda None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:58:13.277922 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-c6b4209b-815a-459f-b32b-19282279abda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:58:13.336163 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1364/2742] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:58:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:58:13.343937 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d028a8-a5c8-4cd3-b17c-ac0e3788fe71 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:13.344236 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d028a8-a5c8-4cd3-b17c-ac0e3788fe71 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:13.344522 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d028a8-a5c8-4cd3-b17c-ac0e3788fe71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:13.358369 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:58:13.363700 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-08d028a8-a5c8-4cd3-b17c-ac0e3788fe71 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:58:13.364895 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-08d028a8-a5c8-4cd3-b17c-ac0e3788fe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:58:13.372911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1379/2743] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:58:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:58:13.377260 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4209b-815a-459f-b32b-19282279abda None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:13.387058 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b4209b-815a-459f-b32b-19282279abda None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:13.391783 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8045fb5-6d03-4578-885f-af4bd8deb34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:13.392479 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8045fb5-6d03-4578-885f-af4bd8deb34e admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:13.392763 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8045fb5-6d03-4578-885f-af4bd8deb34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:13.398054 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8045fb5-6d03-4578-885f-af4bd8deb34e admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:13.398054 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8045fb5-6d03-4578-885f-af4bd8deb34e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 19:58:13.398054 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8045fb5-6d03-4578-885f-af4bd8deb34e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:13.398054 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8045fb5-6d03-4578-885f-af4bd8deb34e admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:13.407753 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1365/2744] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 19:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 19:58:13.411967 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d028a8-a5c8-4cd3-b17c-ac0e3788fe71 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:13.421480 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d028a8-a5c8-4cd3-b17c-ac0e3788fe71 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:13.426718 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554ad163-bd88-4094-b964-bbf0bab2bb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:13.426856 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ad163-bd88-4094-b964-bbf0bab2bb7a admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:13.427035 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ad163-bd88-4094-b964-bbf0bab2bb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:13.427313 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ad163-bd88-4094-b964-bbf0bab2bb7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:13.428484 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554ad163-bd88-4094-b964-bbf0bab2bb7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:13.431063 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554ad163-bd88-4094-b964-bbf0bab2bb7a admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:13.511078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1380/2745] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 19:58:13 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 19:58:13.515173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8045fb5-6d03-4578-885f-af4bd8deb34e admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:13.524919 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:58:13.530289 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8045fb5-6d03-4578-885f-af4bd8deb34e admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:13.586828 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1e9d1-b7cd-41d5-b983-71eff6a9217e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:13.587488 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1e9d1-b7cd-41d5-b983-71eff6a9217e admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:13.587775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1e9d1-b7cd-41d5-b983-71eff6a9217e admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:13.588057 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1e9d1-b7cd-41d5-b983-71eff6a9217e admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:13.588754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1e9d1-b7cd-41d5-b983-71eff6a9217e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:13.590779 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae1e9d1-b7cd-41d5-b983-71eff6a9217e admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:13.657433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1366/2746] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 19:58:13 2022] POST /identity/v3/users => generated 286 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 19:58:13.661652 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554ad163-bd88-4094-b964-bbf0bab2bb7a admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:13.672643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554ad163-bd88-4094-b964-bbf0bab2bb7a admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:13.677221 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140c081-3c18-4626-b0dd-767080f39add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:13.678500 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140c081-3c18-4626-b0dd-767080f39add admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:13.678967 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140c081-3c18-4626-b0dd-767080f39add admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:13.679392 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140c081-3c18-4626-b0dd-767080f39add admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:13.680169 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140c081-3c18-4626-b0dd-767080f39add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:13.681808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140c081-3c18-4626-b0dd-767080f39add admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:13.691533 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1381/2747] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:58:13.695466 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1e9d1-b7cd-41d5-b983-71eff6a9217e admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:13.705157 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1e9d1-b7cd-41d5-b983-71eff6a9217e admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:13.710303 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8c7374-f38a-40ea-8dd3-4b9dc6bf66e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:13.710954 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c7374-f38a-40ea-8dd3-4b9dc6bf66e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:13.711235 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c7374-f38a-40ea-8dd3-4b9dc6bf66e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:13.711512 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8c7374-f38a-40ea-8dd3-4b9dc6bf66e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:13.712266 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c7374-f38a-40ea-8dd3-4b9dc6bf66e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:13.714022 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8c7374-f38a-40ea-8dd3-4b9dc6bf66e8 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:13.724048 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1367/2748] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 19:58:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 19:58:13.728673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140c081-3c18-4626-b0dd-767080f39add admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:13.738417 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140c081-3c18-4626-b0dd-767080f39add admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:13.743654 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebe706-a14d-48d2-8ed0-83fceca6fe57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:13.744529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebe706-a14d-48d2-8ed0-83fceca6fe57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:13.744810 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebe706-a14d-48d2-8ed0-83fceca6fe57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:13.745124 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebe706-a14d-48d2-8ed0-83fceca6fe57 admin admin] PATH_INFO: `/v3/projects/45277feb5e87485d9ab6e905ddc32313/users/0e889514d6aa4fb6bdb6160498757cd9/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:13.759288 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ebe706-a14d-48d2-8ed0-83fceca6fe57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45277feb5e87485d9ab6e905ddc32313, user_id=0e889514d6aa4fb6bdb6160498757cd9, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:13.761981 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ebe706-a14d-48d2-8ed0-83fceca6fe57 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:13.858675 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1382/2749] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 19:58:13 2022] PUT /identity/v3/projects/45277feb5e87485d9ab6e905ddc32313/users/0e889514d6aa4fb6bdb6160498757cd9/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:58:13.870383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d00be-6e62-4174-a08c-775f831add2a None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:13.870679 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d00be-6e62-4174-a08c-775f831add2a None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:13.870954 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d00be-6e62-4174-a08c-775f831add2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:13.897687 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-673d00be-6e62-4174-a08c-775f831add2a None None] MFA Rules not processed for user `0e889514d6aa4fb6bdb6160498757cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:58:13.898889 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-673d00be-6e62-4174-a08c-775f831add2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:58:14.013511 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1368/2750] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:58:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:58:14.024973 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebe706-a14d-48d2-8ed0-83fceca6fe57 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:14.026710 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-c3ebe706-a14d-48d2-8ed0-83fceca6fe57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:58:14.082239 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebe706-a14d-48d2-8ed0-83fceca6fe57 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:14.086761 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a9378-5efd-4897-affd-8bb3d216c825 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:14.087405 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a9378-5efd-4897-affd-8bb3d216c825 service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:14.087696 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a9378-5efd-4897-affd-8bb3d216c825 service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:14.087976 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a9378-5efd-4897-affd-8bb3d216c825 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:14.096142 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a9378-5efd-4897-affd-8bb3d216c825 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:14.097631 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a9378-5efd-4897-affd-8bb3d216c825 service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:14.107219 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1383/2751] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:58:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:58:14.132436 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d00be-6e62-4174-a08c-775f831add2a None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:14.193335 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d00be-6e62-4174-a08c-775f831add2a None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:14.198232 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466f90e-22a4-4d86-8287-59410ffa96fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:14.199040 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466f90e-22a4-4d86-8287-59410ffa96fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:14.199316 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466f90e-22a4-4d86-8287-59410ffa96fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:14.199591 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466f90e-22a4-4d86-8287-59410ffa96fb admin admin] PATH_INFO: `/v3/users/0e889514d6aa4fb6bdb6160498757cd9` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:14.200773 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466f90e-22a4-4d86-8287-59410ffa96fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e889514d6aa4fb6bdb6160498757cd9)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:14.202702 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466f90e-22a4-4d86-8287-59410ffa96fb admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:14.216579 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:58:14.386711 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-2466f90e-22a4-4d86-8287-59410ffa96fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e889514d6aa4fb6bdb6160498757cd9'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:14.399018 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-2466f90e-22a4-4d86-8287-59410ffa96fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e889514d6aa4fb6bdb6160498757cd9'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:14.445751 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-2466f90e-22a4-4d86-8287-59410ffa96fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e889514d6aa4fb6bdb6160498757cd9'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:14.446373 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-2466f90e-22a4-4d86-8287-59410ffa96fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e889514d6aa4fb6bdb6160498757cd9'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:14.446995 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-2466f90e-22a4-4d86-8287-59410ffa96fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e889514d6aa4fb6bdb6160498757cd9'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:14.460208 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1369/2752] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:58:14 2022] DELETE /identity/v3/users/0e889514d6aa4fb6bdb6160498757cd9 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:58:14.464709 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2341c1-be68-41d8-aa8f-78152177bf4d None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:14.465228 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2341c1-be68-41d8-aa8f-78152177bf4d None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:14.465331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2341c1-be68-41d8-aa8f-78152177bf4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:14.483661 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-dd2341c1-be68-41d8-aa8f-78152177bf4d None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 19:58:14.484858 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-dd2341c1-be68-41d8-aa8f-78152177bf4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:58:14.543189 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1384/2753] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 19:58:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 19:58:14.552710 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466f90e-22a4-4d86-8287-59410ffa96fb admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:14.554397 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-2466f90e-22a4-4d86-8287-59410ffa96fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:58:14.611060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466f90e-22a4-4d86-8287-59410ffa96fb admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:14.614730 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14253c6-60cb-40f1-97ae-f705c57c54e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:14.615375 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14253c6-60cb-40f1-97ae-f705c57c54e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:14.615908 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14253c6-60cb-40f1-97ae-f705c57c54e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:14.616197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14253c6-60cb-40f1-97ae-f705c57c54e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:14.625198 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14253c6-60cb-40f1-97ae-f705c57c54e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:14.626643 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14253c6-60cb-40f1-97ae-f705c57c54e5 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:14.636593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1370/2754] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:58:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:58:15.059445 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2341c1-be68-41d8-aa8f-78152177bf4d None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:58:15.061125 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-dd2341c1-be68-41d8-aa8f-78152177bf4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:58:15.074825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2341c1-be68-41d8-aa8f-78152177bf4d None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:58:15.079752 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae70a9a-3454-4d03-b00b-ecc6c62fcc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:58:15.081164 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae70a9a-3454-4d03-b00b-ecc6c62fcc35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:58:15.081456 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae70a9a-3454-4d03-b00b-ecc6c62fcc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:58:15.081887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae70a9a-3454-4d03-b00b-ecc6c62fcc35 admin admin] PATH_INFO: `/v3/projects/45277feb5e87485d9ab6e905ddc32313` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:58:15.083192 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae70a9a-3454-4d03-b00b-ecc6c62fcc35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45277feb5e87485d9ab6e905ddc32313)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:58:15.085452 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae70a9a-3454-4d03-b00b-ecc6c62fcc35 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:58:15.189605 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:58:15.195455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-6ae70a9a-3454-4d03-b00b-ecc6c62fcc35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45277feb5e87485d9ab6e905ddc32313', 'request_id': 'req-6ae70a9a-3454-4d03-b00b-ecc6c62fcc35'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:15.200777 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-6ae70a9a-3454-4d03-b00b-ecc6c62fcc35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45277feb5e87485d9ab6e905ddc32313', 'request_id': 'req-6ae70a9a-3454-4d03-b00b-ecc6c62fcc35'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:15.207146 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-6ae70a9a-3454-4d03-b00b-ecc6c62fcc35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45277feb5e87485d9ab6e905ddc32313', 'request_id': 'req-6ae70a9a-3454-4d03-b00b-ecc6c62fcc35'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:15.279589 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-6ae70a9a-3454-4d03-b00b-ecc6c62fcc35 admin admin] The token cache is being invalidate because project 45277feb5e87485d9ab6e905ddc32313 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:58:15.279871 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-6ae70a9a-3454-4d03-b00b-ecc6c62fcc35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:15.280483 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-6ae70a9a-3454-4d03-b00b-ecc6c62fcc35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:58:15.282094 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1385/2755] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:58:15 2022] DELETE /identity/v3/projects/45277feb5e87485d9ab6e905ddc32313 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:58:24.633035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=47,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:58:25.360527 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=9,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:59:06.432147 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14253c6-60cb-40f1-97ae-f705c57c54e5 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:59:06.433582 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-b14253c6-60cb-40f1-97ae-f705c57c54e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:59:06.504062 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14253c6-60cb-40f1-97ae-f705c57c54e5 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:59:06.568550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d11f2b-a4af-4267-aeb8-2c392cd71456 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2ed13a0c84f128b795002892efc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:59:06.569364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d11f2b-a4af-4267-aeb8-2c392cd71456 service placement] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:59:06.569726 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d11f2b-a4af-4267-aeb8-2c392cd71456 service placement] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:59:06.570079 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d11f2b-a4af-4267-aeb8-2c392cd71456 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:59:06.571926 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-10d11f2b-a4af-4267-aeb8-2c392cd71456 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:59:06.584308 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d11f2b-a4af-4267-aeb8-2c392cd71456 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:59:06.586117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d11f2b-a4af-4267-aeb8-2c392cd71456 service placement] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:59:06.597215 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1371/2756] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 19:59:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 19:59:15.957901 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae70a9a-3454-4d03-b00b-ecc6c62fcc35 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:59:15.959386 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-6ae70a9a-3454-4d03-b00b-ecc6c62fcc35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:59:16.024218 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae70a9a-3454-4d03-b00b-ecc6c62fcc35 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:59:16.028696 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ceef05-0ca1-4c1d-8763-e416362c3d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:59:16.029591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceef05-0ca1-4c1d-8763-e416362c3d95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:59:16.029873 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceef05-0ca1-4c1d-8763-e416362c3d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:59:16.030148 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceef05-0ca1-4c1d-8763-e416362c3d95 admin admin] PATH_INFO: `/v3/users/67ef94416f1741a88c1e18106cad8b2c` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:59:16.031363 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ceef05-0ca1-4c1d-8763-e416362c3d95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67ef94416f1741a88c1e18106cad8b2c)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:59:16.033399 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ceef05-0ca1-4c1d-8763-e416362c3d95 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:59:16.221704 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-38ceef05-0ca1-4c1d-8763-e416362c3d95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67ef94416f1741a88c1e18106cad8b2c'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:59:16.233433 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-38ceef05-0ca1-4c1d-8763-e416362c3d95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67ef94416f1741a88c1e18106cad8b2c'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:59:16.282971 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-38ceef05-0ca1-4c1d-8763-e416362c3d95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67ef94416f1741a88c1e18106cad8b2c'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:59:16.283503 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-38ceef05-0ca1-4c1d-8763-e416362c3d95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67ef94416f1741a88c1e18106cad8b2c'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:59:16.290817 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-38ceef05-0ca1-4c1d-8763-e416362c3d95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67ef94416f1741a88c1e18106cad8b2c'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:59:16.296799 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1386/2757] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:59:15 2022] DELETE /identity/v3/users/67ef94416f1741a88c1e18106cad8b2c => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:59:16.512585 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d11f2b-a4af-4267-aeb8-2c392cd71456 service placement] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:59:16.524571 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d11f2b-a4af-4267-aeb8-2c392cd71456 service placement] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:59:16.587434 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dda64-a690-4fe4-9c41-2791e63beb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:59:16.588297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dda64-a690-4fe4-9c41-2791e63beb3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:59:16.588586 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dda64-a690-4fe4-9c41-2791e63beb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:59:16.588866 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507dda64-a690-4fe4-9c41-2791e63beb3f admin admin] PATH_INFO: `/v3/projects/28f2a07b1f3646209dfdc37dcd2b755e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:59:16.590076 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507dda64-a690-4fe4-9c41-2791e63beb3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=28f2a07b1f3646209dfdc37dcd2b755e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:59:16.591970 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507dda64-a690-4fe4-9c41-2791e63beb3f admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:59:16.684392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:59:16.694036 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-507dda64-a690-4fe4-9c41-2791e63beb3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28f2a07b1f3646209dfdc37dcd2b755e', 'request_id': 'req-507dda64-a690-4fe4-9c41-2791e63beb3f'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:59:16.699273 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-507dda64-a690-4fe4-9c41-2791e63beb3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28f2a07b1f3646209dfdc37dcd2b755e', 'request_id': 'req-507dda64-a690-4fe4-9c41-2791e63beb3f'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:59:16.778771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-507dda64-a690-4fe4-9c41-2791e63beb3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28f2a07b1f3646209dfdc37dcd2b755e', 'request_id': 'req-507dda64-a690-4fe4-9c41-2791e63beb3f'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:59:16.790109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-507dda64-a690-4fe4-9c41-2791e63beb3f admin admin] The token cache is being invalidate because project 28f2a07b1f3646209dfdc37dcd2b755e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:59:16.790397 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-507dda64-a690-4fe4-9c41-2791e63beb3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:59:16.791067 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-507dda64-a690-4fe4-9c41-2791e63beb3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:59:16.792562 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1372/2758] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:59:16 2022] DELETE /identity/v3/projects/28f2a07b1f3646209dfdc37dcd2b755e => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:59:16.796503 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ceef05-0ca1-4c1d-8763-e416362c3d95 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:59:16.798679 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-38ceef05-0ca1-4c1d-8763-e416362c3d95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:59:16.811812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ceef05-0ca1-4c1d-8763-e416362c3d95 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:59:16.867495 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1353f23-e66d-4d78-bcf1-e703d4d51860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:59:16.868370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1353f23-e66d-4d78-bcf1-e703d4d51860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:59:16.868652 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1353f23-e66d-4d78-bcf1-e703d4d51860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:59:16.868930 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1353f23-e66d-4d78-bcf1-e703d4d51860 admin admin] PATH_INFO: `/v3/users/6cb915e3b95d49a08644ac252c9edb6e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:59:16.870145 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1353f23-e66d-4d78-bcf1-e703d4d51860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cb915e3b95d49a08644ac252c9edb6e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:59:16.872068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1353f23-e66d-4d78-bcf1-e703d4d51860 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:59:16.893327 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:59:17.084839 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-b1353f23-e66d-4d78-bcf1-e703d4d51860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cb915e3b95d49a08644ac252c9edb6e'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:59:17.097311 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-b1353f23-e66d-4d78-bcf1-e703d4d51860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cb915e3b95d49a08644ac252c9edb6e'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:59:17.167806 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-b1353f23-e66d-4d78-bcf1-e703d4d51860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cb915e3b95d49a08644ac252c9edb6e'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:59:17.168414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-b1353f23-e66d-4d78-bcf1-e703d4d51860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cb915e3b95d49a08644ac252c9edb6e'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:59:17.177349 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-b1353f23-e66d-4d78-bcf1-e703d4d51860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cb915e3b95d49a08644ac252c9edb6e'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:59:17.184099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1387/2759] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 19:59:16 2022] DELETE /identity/v3/users/6cb915e3b95d49a08644ac252c9edb6e => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:59:17.438711 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dda64-a690-4fe4-9c41-2791e63beb3f admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 19:59:17.440301 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-507dda64-a690-4fe4-9c41-2791e63beb3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 19:59:17.457790 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507dda64-a690-4fe4-9c41-2791e63beb3f admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 19:59:17.518625 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa946ed1-a5a2-4575-8f72-de04bb12e810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 19:59:17.519508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa946ed1-a5a2-4575-8f72-de04bb12e810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 19:59:17.519818 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa946ed1-a5a2-4575-8f72-de04bb12e810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 19:59:17.520101 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa946ed1-a5a2-4575-8f72-de04bb12e810 admin admin] PATH_INFO: `/v3/projects/aefcb4072a0241bca4c1416c113f480a` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 19:59:17.521517 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa946ed1-a5a2-4575-8f72-de04bb12e810 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aefcb4072a0241bca4c1416c113f480a)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 19:59:17.524732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa946ed1-a5a2-4575-8f72-de04bb12e810 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 19:59:17.672591 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-fa946ed1-a5a2-4575-8f72-de04bb12e810 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aefcb4072a0241bca4c1416c113f480a', 'request_id': 'req-fa946ed1-a5a2-4575-8f72-de04bb12e810'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:59:17.678222 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-fa946ed1-a5a2-4575-8f72-de04bb12e810 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aefcb4072a0241bca4c1416c113f480a', 'request_id': 'req-fa946ed1-a5a2-4575-8f72-de04bb12e810'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:59:17.756828 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-fa946ed1-a5a2-4575-8f72-de04bb12e810 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aefcb4072a0241bca4c1416c113f480a', 'request_id': 'req-fa946ed1-a5a2-4575-8f72-de04bb12e810'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:59:17.769386 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-fa946ed1-a5a2-4575-8f72-de04bb12e810 admin admin] The token cache is being invalidate because project aefcb4072a0241bca4c1416c113f480a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 19:59:17.769675 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-fa946ed1-a5a2-4575-8f72-de04bb12e810 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:59:17.770324 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-fa946ed1-a5a2-4575-8f72-de04bb12e810 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 19:59:17.772124 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1373/2760] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 19:59:17 2022] DELETE /identity/v3/projects/aefcb4072a0241bca4c1416c113f480a => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 19:59:27.180482 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:DELETE=7,keystone:SELECT=20,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 19:59:27.765862 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:DELETE=12,keystone:SELECT=62,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:00:30.966572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1353f23-e66d-4d78-bcf1-e703d4d51860 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:00:30.968153 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-b1353f23-e66d-4d78-bcf1-e703d4d51860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:00:31.026256 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1353f23-e66d-4d78-bcf1-e703d4d51860 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:00:31.075554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57af8c-5059-4ed6-b8ec-18f5b5dd63d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:00:31.076233 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57af8c-5059-4ed6-b8ec-18f5b5dd63d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:00:31.076517 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57af8c-5059-4ed6-b8ec-18f5b5dd63d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:00:31.076793 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57af8c-5059-4ed6-b8ec-18f5b5dd63d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:00:31.078845 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-1d57af8c-5059-4ed6-b8ec-18f5b5dd63d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:00:31.148818 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57af8c-5059-4ed6-b8ec-18f5b5dd63d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:00:31.150314 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57af8c-5059-4ed6-b8ec-18f5b5dd63d9 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:00:31.162070 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1388/2761] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:00:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:00:32.333487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa946ed1-a5a2-4575-8f72-de04bb12e810 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:00:32.334888 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-fa946ed1-a5a2-4575-8f72-de04bb12e810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:00:32.347118 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa946ed1-a5a2-4575-8f72-de04bb12e810 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:00:32.350711 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7f1d5-6fcd-4647-b9bd-0bb93c9a81ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:00:32.351362 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7f1d5-6fcd-4647-b9bd-0bb93c9a81ab service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:00:32.351654 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7f1d5-6fcd-4647-b9bd-0bb93c9a81ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:00:32.351928 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a7f1d5-6fcd-4647-b9bd-0bb93c9a81ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:00:32.353782 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-41a7f1d5-6fcd-4647-b9bd-0bb93c9a81ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:00:32.367358 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7f1d5-6fcd-4647-b9bd-0bb93c9a81ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:00:32.368786 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a7f1d5-6fcd-4647-b9bd-0bb93c9a81ab service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:00:32.380337 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1374/2762] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:00:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:00:37.999122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013de448-fe40-4d14-a1ba-f8c0664744df None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:00:37.999649 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013de448-fe40-4d14-a1ba-f8c0664744df None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:00:37.999762 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013de448-fe40-4d14-a1ba-f8c0664744df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:00:38.018022 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-013de448-fe40-4d14-a1ba-f8c0664744df None None] MFA Rules not processed for user `bcdd3f30e42b41308b69e1233686cc17`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:00:38.020485 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-013de448-fe40-4d14-a1ba-f8c0664744df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:00:38.075878 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1389/2763] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:00:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:00:38.373131 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7f1d5-6fcd-4647-b9bd-0bb93c9a81ab service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:00:38.382406 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a7f1d5-6fcd-4647-b9bd-0bb93c9a81ab service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:00:38.386035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2060a-d8d4-4f09-9760-d40eb2084b51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:00:38.386673 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2060a-d8d4-4f09-9760-d40eb2084b51 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:00:38.386961 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2060a-d8d4-4f09-9760-d40eb2084b51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:00:38.387241 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be2060a-d8d4-4f09-9760-d40eb2084b51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:00:38.389021 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-7be2060a-d8d4-4f09-9760-d40eb2084b51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:00:38.399828 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2060a-d8d4-4f09-9760-d40eb2084b51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:00:38.401283 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be2060a-d8d4-4f09-9760-d40eb2084b51 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:00:38.409383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1375/2764] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:00:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:00:38.802657 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013de448-fe40-4d14-a1ba-f8c0664744df None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:00:38.804254 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-013de448-fe40-4d14-a1ba-f8c0664744df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:00:38.809346 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:00:38.877869 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013de448-fe40-4d14-a1ba-f8c0664744df None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:00:38.882996 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52774263-8255-48b4-bbd5-1acbcef8f7f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2ed13a0c84f128b795002892efc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:00:38.883643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52774263-8255-48b4-bbd5-1acbcef8f7f8 service placement] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:00:38.883927 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52774263-8255-48b4-bbd5-1acbcef8f7f8 service placement] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:00:38.884204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52774263-8255-48b4-bbd5-1acbcef8f7f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:00:38.886002 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-52774263-8255-48b4-bbd5-1acbcef8f7f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:00:38.898414 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52774263-8255-48b4-bbd5-1acbcef8f7f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:00:38.900096 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52774263-8255-48b4-bbd5-1acbcef8f7f8 service placement] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:00:38.911017 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1390/2765] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:00:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:00:41.557630 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2060a-d8d4-4f09-9760-d40eb2084b51 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:00:41.559026 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-7be2060a-d8d4-4f09-9760-d40eb2084b51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:00:41.570328 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be2060a-d8d4-4f09-9760-d40eb2084b51 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:00:41.578503 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984b7d3-d25d-472e-9d14-665f7170f97f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:00:41.579227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984b7d3-d25d-472e-9d14-665f7170f97f service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:00:41.579580 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984b7d3-d25d-472e-9d14-665f7170f97f service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:00:41.579925 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984b7d3-d25d-472e-9d14-665f7170f97f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:00:41.581840 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-e984b7d3-d25d-472e-9d14-665f7170f97f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:00:41.592154 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984b7d3-d25d-472e-9d14-665f7170f97f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:00:41.593725 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984b7d3-d25d-472e-9d14-665f7170f97f service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:00:41.602048 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1376/2766] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:00:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:00:48.905942 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=31 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:00:51.598231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=18 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:01:46.966756 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52774263-8255-48b4-bbd5-1acbcef8f7f8 service placement] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:01:46.968362 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-52774263-8255-48b4-bbd5-1acbcef8f7f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:01:47.189411 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52774263-8255-48b4-bbd5-1acbcef8f7f8 service placement] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:01:47.201753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab659a20-51c7-4444-937e-a401349c1665 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:01:47.201753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab659a20-51c7-4444-937e-a401349c1665 service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:01:47.201753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab659a20-51c7-4444-937e-a401349c1665 service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:01:47.201753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab659a20-51c7-4444-937e-a401349c1665 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:01:47.202886 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ab659a20-51c7-4444-937e-a401349c1665 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:01:47.217839 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab659a20-51c7-4444-937e-a401349c1665 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:01:47.219435 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab659a20-51c7-4444-937e-a401349c1665 service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:01:47.230887 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1391/2767] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:01:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:01:57.170402 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984b7d3-d25d-472e-9d14-665f7170f97f service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:01:57.175047 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-e984b7d3-d25d-472e-9d14-665f7170f97f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:01:57.225746 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=28 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:01:57.285266 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984b7d3-d25d-472e-9d14-665f7170f97f service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:01:57.289853 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d756f381-6081-4578-a4f6-688fa81dfd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:01:57.290691 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756f381-6081-4578-a4f6-688fa81dfd19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:01:57.290980 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756f381-6081-4578-a4f6-688fa81dfd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:01:57.291257 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d756f381-6081-4578-a4f6-688fa81dfd19 admin admin] PATH_INFO: `/v3/users/bcdd3f30e42b41308b69e1233686cc17` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:01:57.293150 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756f381-6081-4578-a4f6-688fa81dfd19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcdd3f30e42b41308b69e1233686cc17)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:01:57.295055 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d756f381-6081-4578-a4f6-688fa81dfd19 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:01:57.486919 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d756f381-6081-4578-a4f6-688fa81dfd19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcdd3f30e42b41308b69e1233686cc17'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:01:57.498717 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d756f381-6081-4578-a4f6-688fa81dfd19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcdd3f30e42b41308b69e1233686cc17'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:01:57.580903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d756f381-6081-4578-a4f6-688fa81dfd19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcdd3f30e42b41308b69e1233686cc17'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:01:57.581549 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d756f381-6081-4578-a4f6-688fa81dfd19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcdd3f30e42b41308b69e1233686cc17'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:01:57.582150 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-d756f381-6081-4578-a4f6-688fa81dfd19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcdd3f30e42b41308b69e1233686cc17'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:01:57.595597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1377/2768] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 20:01:57 2022] DELETE /identity/v3/users/bcdd3f30e42b41308b69e1233686cc17 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 20:01:57.870331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab659a20-51c7-4444-937e-a401349c1665 service nova] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:01:57.872915 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab659a20-51c7-4444-937e-a401349c1665 service nova] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:01:57.930207 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c201ec-5739-43a2-b335-592a146ffdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:01:57.931068 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c201ec-5739-43a2-b335-592a146ffdbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:01:57.931352 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c201ec-5739-43a2-b335-592a146ffdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:01:57.931629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c201ec-5739-43a2-b335-592a146ffdbb admin admin] PATH_INFO: `/v3/projects/51d8222fe16e403999de6fc5803b50d3` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:01:57.932811 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c201ec-5739-43a2-b335-592a146ffdbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=51d8222fe16e403999de6fc5803b50d3)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:01:57.935661 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c201ec-5739-43a2-b335-592a146ffdbb admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:01:58.055039 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-51c201ec-5739-43a2-b335-592a146ffdbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51d8222fe16e403999de6fc5803b50d3', 'request_id': 'req-51c201ec-5739-43a2-b335-592a146ffdbb'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:01:58.061209 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-51c201ec-5739-43a2-b335-592a146ffdbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51d8222fe16e403999de6fc5803b50d3', 'request_id': 'req-51c201ec-5739-43a2-b335-592a146ffdbb'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:01:58.067250 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-51c201ec-5739-43a2-b335-592a146ffdbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51d8222fe16e403999de6fc5803b50d3', 'request_id': 'req-51c201ec-5739-43a2-b335-592a146ffdbb'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:01:58.157753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-51c201ec-5739-43a2-b335-592a146ffdbb admin admin] The token cache is being invalidate because project 51d8222fe16e403999de6fc5803b50d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 20:01:58.158040 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-51c201ec-5739-43a2-b335-592a146ffdbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:01:58.158680 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-51c201ec-5739-43a2-b335-592a146ffdbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:01:58.160295 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1392/2769] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 20:01:57 2022] DELETE /identity/v3/projects/51d8222fe16e403999de6fc5803b50d3 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:01:58.165378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef4405-057c-48fb-8c44-0b1644392515 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:01:58.170072 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef4405-057c-48fb-8c44-0b1644392515 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:01:58.170072 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef4405-057c-48fb-8c44-0b1644392515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:01:58.223891 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:01:58.226990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-43ef4405-057c-48fb-8c44-0b1644392515 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:01:58.228186 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-43ef4405-057c-48fb-8c44-0b1644392515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:01:58.286670 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1378/2770] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:01:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:01:58.295908 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d00927-1020-4dbd-9dc1-05e122e69b26 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:01:58.296430 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d00927-1020-4dbd-9dc1-05e122e69b26 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:01:58.296637 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d00927-1020-4dbd-9dc1-05e122e69b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:01:58.318131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-68d00927-1020-4dbd-9dc1-05e122e69b26 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:01:58.318131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-68d00927-1020-4dbd-9dc1-05e122e69b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:01:58.325552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1393/2771] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:01:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:01:58.329931 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef4405-057c-48fb-8c44-0b1644392515 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:01:58.340420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef4405-057c-48fb-8c44-0b1644392515 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:01:58.346894 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482fec6-5bf8-4a6a-ac78-41bd145a2c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:01:58.346894 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482fec6-5bf8-4a6a-ac78-41bd145a2c3c admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:01:58.346894 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482fec6-5bf8-4a6a-ac78-41bd145a2c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:01:58.346894 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482fec6-5bf8-4a6a-ac78-41bd145a2c3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:01:58.347350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482fec6-5bf8-4a6a-ac78-41bd145a2c3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 20:01:58.347618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482fec6-5bf8-4a6a-ac78-41bd145a2c3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:01:58.349229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482fec6-5bf8-4a6a-ac78-41bd145a2c3c admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:01:58.361232 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1379/2772] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 20:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 20:01:58.365552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d00927-1020-4dbd-9dc1-05e122e69b26 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:01:58.374785 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d00927-1020-4dbd-9dc1-05e122e69b26 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:01:58.379806 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422fb24e-06ba-4276-8c3f-bc28a6d50ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:01:58.380455 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422fb24e-06ba-4276-8c3f-bc28a6d50ba1 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:01:58.380800 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422fb24e-06ba-4276-8c3f-bc28a6d50ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:01:58.381122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422fb24e-06ba-4276-8c3f-bc28a6d50ba1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:01:58.381830 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422fb24e-06ba-4276-8c3f-bc28a6d50ba1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:01:58.383737 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422fb24e-06ba-4276-8c3f-bc28a6d50ba1 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:01:58.497185 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1394/2773] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 20:01:58 2022] POST /identity/v3/projects => generated 363 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 20:01:58.498350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482fec6-5bf8-4a6a-ac78-41bd145a2c3c admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:01:58.508452 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482fec6-5bf8-4a6a-ac78-41bd145a2c3c admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:01:58.566665 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b19fa-aeb3-40fe-9103-9ec2a26fabda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:01:58.567334 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b19fa-aeb3-40fe-9103-9ec2a26fabda admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:01:58.567618 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b19fa-aeb3-40fe-9103-9ec2a26fabda admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:01:58.567896 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b19fa-aeb3-40fe-9103-9ec2a26fabda admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:01:58.568627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b19fa-aeb3-40fe-9103-9ec2a26fabda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:01:58.570510 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b19fa-aeb3-40fe-9103-9ec2a26fabda admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:01:58.675754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1380/2774] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 20:01:58 2022] POST /identity/v3/users => generated 282 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 20:01:58.679922 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422fb24e-06ba-4276-8c3f-bc28a6d50ba1 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:01:58.690212 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422fb24e-06ba-4276-8c3f-bc28a6d50ba1 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:01:58.694829 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372021d9-41fa-4d25-ace1-1ba1392288ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:01:58.695482 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372021d9-41fa-4d25-ace1-1ba1392288ac admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:01:58.695768 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372021d9-41fa-4d25-ace1-1ba1392288ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:01:58.696046 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372021d9-41fa-4d25-ace1-1ba1392288ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:01:58.696767 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372021d9-41fa-4d25-ace1-1ba1392288ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:01:58.698426 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372021d9-41fa-4d25-ace1-1ba1392288ac admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:01:58.709456 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1395/2775] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 20:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 20:01:58.713273 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b19fa-aeb3-40fe-9103-9ec2a26fabda admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:01:58.724208 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b19fa-aeb3-40fe-9103-9ec2a26fabda admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:01:58.728854 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cfd6be-e131-47db-aa80-65b35c1c903a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:01:58.729543 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfd6be-e131-47db-aa80-65b35c1c903a admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:01:58.729835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfd6be-e131-47db-aa80-65b35c1c903a admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:01:58.730119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfd6be-e131-47db-aa80-65b35c1c903a admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:01:58.731103 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfd6be-e131-47db-aa80-65b35c1c903a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:01:58.732759 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfd6be-e131-47db-aa80-65b35c1c903a admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:01:58.742546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1381/2776] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 20:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 20:01:58.746511 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372021d9-41fa-4d25-ace1-1ba1392288ac admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:01:58.756231 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372021d9-41fa-4d25-ace1-1ba1392288ac admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:01:58.760753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071b2aa-c46c-40fc-8275-28cb5d2aab5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:01:58.762185 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071b2aa-c46c-40fc-8275-28cb5d2aab5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:01:58.762674 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071b2aa-c46c-40fc-8275-28cb5d2aab5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:01:58.763043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071b2aa-c46c-40fc-8275-28cb5d2aab5e admin admin] PATH_INFO: `/v3/projects/65af9ff2b4e0457985d41d8c08157636/users/87209d40d576480c987027217d5fedc3/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:01:58.777267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d071b2aa-c46c-40fc-8275-28cb5d2aab5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=65af9ff2b4e0457985d41d8c08157636, user_id=87209d40d576480c987027217d5fedc3, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:01:58.780629 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d071b2aa-c46c-40fc-8275-28cb5d2aab5e admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:01:58.847514 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1396/2777] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 20:01:58 2022] PUT /identity/v3/projects/65af9ff2b4e0457985d41d8c08157636/users/87209d40d576480c987027217d5fedc3/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:01:58.857457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd60074-66ec-4863-ad3f-6956723e3819 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:01:58.857805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd60074-66ec-4863-ad3f-6956723e3819 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:01:58.858073 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd60074-66ec-4863-ad3f-6956723e3819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:01:58.885281 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-2bd60074-66ec-4863-ad3f-6956723e3819 None None] MFA Rules not processed for user `87209d40d576480c987027217d5fedc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:01:58.886568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-2bd60074-66ec-4863-ad3f-6956723e3819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:01:58.952936 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1382/2778] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:01:58 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 20:01:58.957460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071b2aa-c46c-40fc-8275-28cb5d2aab5e admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:01:58.986878 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:01:59.019324 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071b2aa-c46c-40fc-8275-28cb5d2aab5e admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:01:59.023892 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66776244-61d3-42fe-b0da-58d7b40b8c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:01:59.024543 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66776244-61d3-42fe-b0da-58d7b40b8c94 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:01:59.024910 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66776244-61d3-42fe-b0da-58d7b40b8c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:01:59.025287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66776244-61d3-42fe-b0da-58d7b40b8c94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:01:59.026373 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66776244-61d3-42fe-b0da-58d7b40b8c94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:01:59.028896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66776244-61d3-42fe-b0da-58d7b40b8c94 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:01:59.117355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1397/2779] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 20:01:58 2022] POST /identity/v3/projects => generated 363 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 20:01:59.121640 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd60074-66ec-4863-ad3f-6956723e3819 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:01:59.127060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:01:59.132620 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd60074-66ec-4863-ad3f-6956723e3819 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:01:59.189117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d0c71-b98f-412e-9172-989b60b61d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:01:59.189793 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d0c71-b98f-412e-9172-989b60b61d41 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:01:59.190080 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d0c71-b98f-412e-9172-989b60b61d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:01:59.190477 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d0c71-b98f-412e-9172-989b60b61d41 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:01:59.191173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d0c71-b98f-412e-9172-989b60b61d41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:01:59.193127 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22d0c71-b98f-412e-9172-989b60b61d41 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:01:59.296962 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1383/2780] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 20:01:59 2022] POST /identity/v3/users => generated 288 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 20:01:59.300886 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66776244-61d3-42fe-b0da-58d7b40b8c94 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:01:59.312585 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66776244-61d3-42fe-b0da-58d7b40b8c94 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:01:59.317638 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dcfdbe-749e-4302-9e2b-c6da1ca4ad38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:01:59.318304 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcfdbe-749e-4302-9e2b-c6da1ca4ad38 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:01:59.318596 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcfdbe-749e-4302-9e2b-c6da1ca4ad38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:01:59.318877 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcfdbe-749e-4302-9e2b-c6da1ca4ad38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:01:59.319631 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dcfdbe-749e-4302-9e2b-c6da1ca4ad38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:01:59.321521 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dcfdbe-749e-4302-9e2b-c6da1ca4ad38 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:01:59.332122 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1398/2781] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 20:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 20:01:59.336358 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d0c71-b98f-412e-9172-989b60b61d41 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:01:59.346746 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22d0c71-b98f-412e-9172-989b60b61d41 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:01:59.352159 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4498d-e673-4e17-8b21-735409609ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:01:59.353983 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4498d-e673-4e17-8b21-735409609ffb admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:01:59.354534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4498d-e673-4e17-8b21-735409609ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:01:59.355066 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd4498d-e673-4e17-8b21-735409609ffb admin admin] PATH_INFO: `/v3/projects/8f2e173601ae43a1a22b03058b31ee78/users/8c932db1b7a74ff78772a37ef421a968/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:01:59.372928 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4498d-e673-4e17-8b21-735409609ffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f2e173601ae43a1a22b03058b31ee78, user_id=8c932db1b7a74ff78772a37ef421a968, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:01:59.375650 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd4498d-e673-4e17-8b21-735409609ffb admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:01:59.432081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1384/2782] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 20:01:59 2022] PUT /identity/v3/projects/8f2e173601ae43a1a22b03058b31ee78/users/8c932db1b7a74ff78772a37ef421a968/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:01:59.436746 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dcfdbe-749e-4302-9e2b-c6da1ca4ad38 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:01:59.446584 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dcfdbe-749e-4302-9e2b-c6da1ca4ad38 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:01:59.502613 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e8a48c-cd9b-43c9-bad7-494bad359ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:01:59.503283 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8a48c-cd9b-43c9-bad7-494bad359ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:01:59.503572 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8a48c-cd9b-43c9-bad7-494bad359ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:01:59.503848 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e8a48c-cd9b-43c9-bad7-494bad359ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:01:59.504563 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e8a48c-cd9b-43c9-bad7-494bad359ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:01:59.506226 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e8a48c-cd9b-43c9-bad7-494bad359ea7 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:01:59.521850 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1399/2783] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 20:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 20:01:59.523020 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4498d-e673-4e17-8b21-735409609ffb admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:01:59.533568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd4498d-e673-4e17-8b21-735409609ffb admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:01:59.538341 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0dccd-2ebf-4108-b19b-77ac9ca8b660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:01:59.539268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0dccd-2ebf-4108-b19b-77ac9ca8b660 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:01:59.539550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0dccd-2ebf-4108-b19b-77ac9ca8b660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:01:59.539824 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0dccd-2ebf-4108-b19b-77ac9ca8b660 admin admin] PATH_INFO: `/v3/projects/8f2e173601ae43a1a22b03058b31ee78/users/8c932db1b7a74ff78772a37ef421a968/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:01:59.541874 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0dccd-2ebf-4108-b19b-77ac9ca8b660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f2e173601ae43a1a22b03058b31ee78, user_id=8c932db1b7a74ff78772a37ef421a968, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:01:59.545929 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0dccd-2ebf-4108-b19b-77ac9ca8b660 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:01:59.556484 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1385/2784] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 20:01:59 2022] PUT /identity/v3/projects/8f2e173601ae43a1a22b03058b31ee78/users/8c932db1b7a74ff78772a37ef421a968/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:01:59.568749 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71147e8f-e8bf-4b3d-bc2f-772a23cc1330 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:01:59.569071 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71147e8f-e8bf-4b3d-bc2f-772a23cc1330 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:01:59.569356 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71147e8f-e8bf-4b3d-bc2f-772a23cc1330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:01:59.596988 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-71147e8f-e8bf-4b3d-bc2f-772a23cc1330 None None] MFA Rules not processed for user `8c932db1b7a74ff78772a37ef421a968`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:01:59.598183 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-71147e8f-e8bf-4b3d-bc2f-772a23cc1330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:01:59.665178 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1400/2785] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:01:59 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:01:59.677010 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0dccd-2ebf-4108-b19b-77ac9ca8b660 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:01:59.678689 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-2eb0dccd-2ebf-4108-b19b-77ac9ca8b660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:01:59.734466 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0dccd-2ebf-4108-b19b-77ac9ca8b660 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:01:59.738070 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20710f71-142d-4a34-8d1f-db61bfb11082 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:01:59.738837 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20710f71-142d-4a34-8d1f-db61bfb11082 service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:01:59.739123 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20710f71-142d-4a34-8d1f-db61bfb11082 service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:01:59.739397 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20710f71-142d-4a34-8d1f-db61bfb11082 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:01:59.748382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20710f71-142d-4a34-8d1f-db61bfb11082 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:01:59.749812 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20710f71-142d-4a34-8d1f-db61bfb11082 service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:01:59.758852 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1386/2786] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:01:59 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:01:59.924649 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71147e8f-e8bf-4b3d-bc2f-772a23cc1330 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:01:59.934544 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71147e8f-e8bf-4b3d-bc2f-772a23cc1330 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:01:59.938103 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e81b5a6-d31d-4238-810a-0136fac2f5cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:01:59.938742 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81b5a6-d31d-4238-810a-0136fac2f5cf service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:01:59.939027 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81b5a6-d31d-4238-810a-0136fac2f5cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:01:59.939308 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e81b5a6-d31d-4238-810a-0136fac2f5cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:01:59.958971 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:01:59.996706 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81b5a6-d31d-4238-810a-0136fac2f5cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:01:59.998208 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e81b5a6-d31d-4238-810a-0136fac2f5cf service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:00.007462 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1401/2787] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:01:59 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:02:03.451211 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20710f71-142d-4a34-8d1f-db61bfb11082 service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:03.507709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:02:03.512068 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20710f71-142d-4a34-8d1f-db61bfb11082 service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:03.516614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ebf2f1-81b5-43a3-ba51-d8ccf1f67e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:03.547263 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ebf2f1-81b5-43a3-ba51-d8ccf1f67e62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:03.547263 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ebf2f1-81b5-43a3-ba51-d8ccf1f67e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:03.547263 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ebf2f1-81b5-43a3-ba51-d8ccf1f67e62 admin admin] PATH_INFO: `/v3/users/87209d40d576480c987027217d5fedc3` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:03.547263 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ebf2f1-81b5-43a3-ba51-d8ccf1f67e62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87209d40d576480c987027217d5fedc3)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:03.547263 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ebf2f1-81b5-43a3-ba51-d8ccf1f67e62 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:03.702651 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-40ebf2f1-81b5-43a3-ba51-d8ccf1f67e62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87209d40d576480c987027217d5fedc3'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:03.714305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-40ebf2f1-81b5-43a3-ba51-d8ccf1f67e62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87209d40d576480c987027217d5fedc3'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:03.768649 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-40ebf2f1-81b5-43a3-ba51-d8ccf1f67e62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87209d40d576480c987027217d5fedc3'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:03.769236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-40ebf2f1-81b5-43a3-ba51-d8ccf1f67e62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87209d40d576480c987027217d5fedc3'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:03.769868 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-40ebf2f1-81b5-43a3-ba51-d8ccf1f67e62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87209d40d576480c987027217d5fedc3'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:03.783000 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1387/2788] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 20:02:03 2022] DELETE /identity/v3/users/87209d40d576480c987027217d5fedc3 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:02:03.787473 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c72901-a900-4cd0-bb17-da46b204ac5b None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:03.787966 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c72901-a900-4cd0-bb17-da46b204ac5b None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:03.788087 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c72901-a900-4cd0-bb17-da46b204ac5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:03.804796 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-f2c72901-a900-4cd0-bb17-da46b204ac5b None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:02:03.806043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-f2c72901-a900-4cd0-bb17-da46b204ac5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:03.863643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1402/2789] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:02:03.873272 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ebf2f1-81b5-43a3-ba51-d8ccf1f67e62 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:03.875164 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-40ebf2f1-81b5-43a3-ba51-d8ccf1f67e62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:03.955605 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ebf2f1-81b5-43a3-ba51-d8ccf1f67e62 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:03.959230 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87adc2d1-5655-4f3c-82d3-31f2e9f4a9e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:03.959872 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adc2d1-5655-4f3c-82d3-31f2e9f4a9e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:03.960155 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adc2d1-5655-4f3c-82d3-31f2e9f4a9e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:03.960432 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87adc2d1-5655-4f3c-82d3-31f2e9f4a9e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:03.970041 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87adc2d1-5655-4f3c-82d3-31f2e9f4a9e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:03.971457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87adc2d1-5655-4f3c-82d3-31f2e9f4a9e6 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:03.980768 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1388/2790] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:02:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:02:04.353438 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c72901-a900-4cd0-bb17-da46b204ac5b None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:04.359963 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-f2c72901-a900-4cd0-bb17-da46b204ac5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:04.367019 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c72901-a900-4cd0-bb17-da46b204ac5b None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:04.371938 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b906f-0284-459c-b72b-4c37b42450af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:04.373552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b906f-0284-459c-b72b-4c37b42450af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:04.373881 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b906f-0284-459c-b72b-4c37b42450af admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:04.374161 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b906f-0284-459c-b72b-4c37b42450af admin admin] PATH_INFO: `/v3/projects/65af9ff2b4e0457985d41d8c08157636` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:04.375451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b906f-0284-459c-b72b-4c37b42450af admin admin] RBAC: Authorizing `identity:delete_project(project_id=65af9ff2b4e0457985d41d8c08157636)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:04.377456 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78b906f-0284-459c-b72b-4c37b42450af admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:04.513385 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-e78b906f-0284-459c-b72b-4c37b42450af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65af9ff2b4e0457985d41d8c08157636', 'request_id': 'req-e78b906f-0284-459c-b72b-4c37b42450af'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:04.518907 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-e78b906f-0284-459c-b72b-4c37b42450af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65af9ff2b4e0457985d41d8c08157636', 'request_id': 'req-e78b906f-0284-459c-b72b-4c37b42450af'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:04.525115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-e78b906f-0284-459c-b72b-4c37b42450af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65af9ff2b4e0457985d41d8c08157636', 'request_id': 'req-e78b906f-0284-459c-b72b-4c37b42450af'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:04.603221 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-e78b906f-0284-459c-b72b-4c37b42450af admin admin] The token cache is being invalidate because project 65af9ff2b4e0457985d41d8c08157636 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 20:02:04.603418 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-e78b906f-0284-459c-b72b-4c37b42450af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:04.604040 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-e78b906f-0284-459c-b72b-4c37b42450af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:04.605618 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1403/2791] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 20:02:04 2022] DELETE /identity/v3/projects/65af9ff2b4e0457985d41d8c08157636 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:02:04.610996 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87adc2d1-5655-4f3c-82d3-31f2e9f4a9e6 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:04.620555 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87adc2d1-5655-4f3c-82d3-31f2e9f4a9e6 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:04.676819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6c0d8-3d05-453a-beec-984387f12212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:04.677712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6c0d8-3d05-453a-beec-984387f12212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:04.677999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6c0d8-3d05-453a-beec-984387f12212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:04.678273 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6c0d8-3d05-453a-beec-984387f12212 admin admin] PATH_INFO: `/v3/users/8c932db1b7a74ff78772a37ef421a968` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:04.679439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6c0d8-3d05-453a-beec-984387f12212 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c932db1b7a74ff78772a37ef421a968)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:04.681387 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6c0d8-3d05-453a-beec-984387f12212 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:04.682850 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:02:04.845313 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-7ec6c0d8-3d05-453a-beec-984387f12212 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c932db1b7a74ff78772a37ef421a968'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:04.858595 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-7ec6c0d8-3d05-453a-beec-984387f12212 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c932db1b7a74ff78772a37ef421a968'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:04.928526 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-7ec6c0d8-3d05-453a-beec-984387f12212 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c932db1b7a74ff78772a37ef421a968'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:04.929182 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-7ec6c0d8-3d05-453a-beec-984387f12212 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c932db1b7a74ff78772a37ef421a968'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:04.929790 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-7ec6c0d8-3d05-453a-beec-984387f12212 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c932db1b7a74ff78772a37ef421a968'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:04.942495 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1389/2792] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 20:02:04 2022] DELETE /identity/v3/users/8c932db1b7a74ff78772a37ef421a968 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:02:05.330728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b906f-0284-459c-b72b-4c37b42450af admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:05.332227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-e78b906f-0284-459c-b72b-4c37b42450af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:05.346044 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b906f-0284-459c-b72b-4c37b42450af admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:05.389947 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:02:05.404575 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07242dc-71b9-474e-82f6-8f4b35ea3300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:05.405406 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07242dc-71b9-474e-82f6-8f4b35ea3300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:05.405688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07242dc-71b9-474e-82f6-8f4b35ea3300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:05.405960 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07242dc-71b9-474e-82f6-8f4b35ea3300 admin admin] PATH_INFO: `/v3/projects/8f2e173601ae43a1a22b03058b31ee78` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:05.407130 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07242dc-71b9-474e-82f6-8f4b35ea3300 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f2e173601ae43a1a22b03058b31ee78)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:05.409131 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07242dc-71b9-474e-82f6-8f4b35ea3300 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:05.506966 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-a07242dc-71b9-474e-82f6-8f4b35ea3300 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f2e173601ae43a1a22b03058b31ee78', 'request_id': 'req-a07242dc-71b9-474e-82f6-8f4b35ea3300'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:05.512523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-a07242dc-71b9-474e-82f6-8f4b35ea3300 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f2e173601ae43a1a22b03058b31ee78', 'request_id': 'req-a07242dc-71b9-474e-82f6-8f4b35ea3300'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:05.519230 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-a07242dc-71b9-474e-82f6-8f4b35ea3300 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f2e173601ae43a1a22b03058b31ee78', 'request_id': 'req-a07242dc-71b9-474e-82f6-8f4b35ea3300'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:05.579123 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-a07242dc-71b9-474e-82f6-8f4b35ea3300 admin admin] The token cache is being invalidate because project 8f2e173601ae43a1a22b03058b31ee78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 20:02:05.579408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-a07242dc-71b9-474e-82f6-8f4b35ea3300 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:05.580041 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-a07242dc-71b9-474e-82f6-8f4b35ea3300 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:05.581723 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1404/2793] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 20:02:05 2022] DELETE /identity/v3/projects/8f2e173601ae43a1a22b03058b31ee78 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:02:05.586739 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2216ee-c622-41e2-85b0-94bedd24a1e8 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:05.587050 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2216ee-c622-41e2-85b0-94bedd24a1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:05.587334 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2216ee-c622-41e2-85b0-94bedd24a1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:05.606042 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-fc2216ee-c622-41e2-85b0-94bedd24a1e8 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:02:05.607579 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-fc2216ee-c622-41e2-85b0-94bedd24a1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:05.668244 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1390/2794] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:02:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:02:05.676082 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba0757-09e5-4c15-b6db-a050d82af68c None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:05.676371 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba0757-09e5-4c15-b6db-a050d82af68c None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:05.676654 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba0757-09e5-4c15-b6db-a050d82af68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:05.695015 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-79ba0757-09e5-4c15-b6db-a050d82af68c None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:02:05.696204 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-79ba0757-09e5-4c15-b6db-a050d82af68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:05.704751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1405/2795] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:02:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:02:05.708811 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2216ee-c622-41e2-85b0-94bedd24a1e8 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:05.718420 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2216ee-c622-41e2-85b0-94bedd24a1e8 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:05.722991 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe37c3-fb29-438c-a0ec-d5de8c314c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:05.723643 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe37c3-fb29-438c-a0ec-d5de8c314c9c admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:05.723932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe37c3-fb29-438c-a0ec-d5de8c314c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:05.724212 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe37c3-fb29-438c-a0ec-d5de8c314c9c admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:05.724961 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe37c3-fb29-438c-a0ec-d5de8c314c9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 20:02:05.725288 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe37c3-fb29-438c-a0ec-d5de8c314c9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:05.726919 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe37c3-fb29-438c-a0ec-d5de8c314c9c admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:05.738504 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1391/2796] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 20:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 20:02:05.742546 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba0757-09e5-4c15-b6db-a050d82af68c None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:05.752114 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba0757-09e5-4c15-b6db-a050d82af68c None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:05.756756 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53b9ba-e511-4585-9c3c-6ad33b38273a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:05.757434 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53b9ba-e511-4585-9c3c-6ad33b38273a admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:05.757751 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53b9ba-e511-4585-9c3c-6ad33b38273a admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:05.758030 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53b9ba-e511-4585-9c3c-6ad33b38273a admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:05.758717 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f53b9ba-e511-4585-9c3c-6ad33b38273a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:05.760622 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f53b9ba-e511-4585-9c3c-6ad33b38273a admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:05.842753 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1406/2797] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 20:02:05 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 20:02:05.846791 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe37c3-fb29-438c-a0ec-d5de8c314c9c admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:05.856576 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe37c3-fb29-438c-a0ec-d5de8c314c9c admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:05.914555 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d83e20-e983-4428-9b4f-250b893a277d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:05.915224 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d83e20-e983-4428-9b4f-250b893a277d admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:05.915511 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d83e20-e983-4428-9b4f-250b893a277d admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:05.915790 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d83e20-e983-4428-9b4f-250b893a277d admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:05.916475 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d83e20-e983-4428-9b4f-250b893a277d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:05.918472 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d83e20-e983-4428-9b4f-250b893a277d admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:05.928415 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:02:05.995749 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1392/2798] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 20:02:05 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 20:02:06.000726 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53b9ba-e511-4585-9c3c-6ad33b38273a admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:06.010914 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53b9ba-e511-4585-9c3c-6ad33b38273a admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:06.015851 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b428b0-3578-45ea-9fe5-4adbb357c7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:06.016508 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b428b0-3578-45ea-9fe5-4adbb357c7d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:06.016796 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b428b0-3578-45ea-9fe5-4adbb357c7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:06.017112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b428b0-3578-45ea-9fe5-4adbb357c7d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:06.017952 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b428b0-3578-45ea-9fe5-4adbb357c7d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:06.019552 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b428b0-3578-45ea-9fe5-4adbb357c7d5 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:06.030067 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1407/2799] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 20:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 20:02:06.033974 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d83e20-e983-4428-9b4f-250b893a277d admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:06.043857 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d83e20-e983-4428-9b4f-250b893a277d admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:06.049567 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331111a-e072-4010-b552-0e930861e8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:06.050227 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331111a-e072-4010-b552-0e930861e8db admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:06.050511 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331111a-e072-4010-b552-0e930861e8db admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:06.050786 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331111a-e072-4010-b552-0e930861e8db admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:06.051507 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331111a-e072-4010-b552-0e930861e8db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:06.053168 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331111a-e072-4010-b552-0e930861e8db admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:06.064534 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1393/2800] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 20:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 20:02:06.068529 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b428b0-3578-45ea-9fe5-4adbb357c7d5 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:06.078926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b428b0-3578-45ea-9fe5-4adbb357c7d5 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:06.083559 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb0540-38aa-45d3-8693-7b1247d7ebd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:06.084441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb0540-38aa-45d3-8693-7b1247d7ebd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:06.084728 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb0540-38aa-45d3-8693-7b1247d7ebd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:06.085048 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adb0540-38aa-45d3-8693-7b1247d7ebd9 admin admin] PATH_INFO: `/v3/projects/69730ec093f74ebaaebc8c6980ff6f6b/users/8e8193cb9fb74e1aa235f5e54aec7edc/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:06.099667 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adb0540-38aa-45d3-8693-7b1247d7ebd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69730ec093f74ebaaebc8c6980ff6f6b, user_id=8e8193cb9fb74e1aa235f5e54aec7edc, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:06.102441 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adb0540-38aa-45d3-8693-7b1247d7ebd9 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:06.197195 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1408/2801] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 20:02:06 2022] PUT /identity/v3/projects/69730ec093f74ebaaebc8c6980ff6f6b/users/8e8193cb9fb74e1aa235f5e54aec7edc/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:02:06.207919 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c1a94-f051-4b11-9a51-8efbec2ed402 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:06.208211 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c1a94-f051-4b11-9a51-8efbec2ed402 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:06.208492 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686c1a94-f051-4b11-9a51-8efbec2ed402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:06.234679 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-686c1a94-f051-4b11-9a51-8efbec2ed402 None None] MFA Rules not processed for user `8e8193cb9fb74e1aa235f5e54aec7edc`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:02:06.235869 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-686c1a94-f051-4b11-9a51-8efbec2ed402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:06.300029 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1394/2802] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:02:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:02:06.304396 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb0540-38aa-45d3-8693-7b1247d7ebd9 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:06.367774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adb0540-38aa-45d3-8693-7b1247d7ebd9 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:06.372434 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf559a2a-974b-440d-8032-7acb26074fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:06.373357 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf559a2a-974b-440d-8032-7acb26074fab admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:06.373643 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf559a2a-974b-440d-8032-7acb26074fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:06.373925 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf559a2a-974b-440d-8032-7acb26074fab admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:06.374604 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf559a2a-974b-440d-8032-7acb26074fab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:06.376545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf559a2a-974b-440d-8032-7acb26074fab admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:06.390464 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:02:06.518952 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1409/2803] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 20:02:06 2022] POST /identity/v3/projects => generated 379 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 20:02:06.523125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c1a94-f051-4b11-9a51-8efbec2ed402 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:06.534890 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686c1a94-f051-4b11-9a51-8efbec2ed402 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:06.589801 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574796b-20b7-4d03-ace2-cc7beac4ddc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:06.590481 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574796b-20b7-4d03-ace2-cc7beac4ddc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:06.590762 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574796b-20b7-4d03-ace2-cc7beac4ddc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:06.591051 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9574796b-20b7-4d03-ace2-cc7beac4ddc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:06.592225 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9574796b-20b7-4d03-ace2-cc7beac4ddc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:06.594177 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9574796b-20b7-4d03-ace2-cc7beac4ddc6 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:06.689326 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1395/2804] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 20:02:06 2022] POST /identity/v3/users => generated 296 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 20:02:06.693227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf559a2a-974b-440d-8032-7acb26074fab admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:06.704166 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf559a2a-974b-440d-8032-7acb26074fab admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:06.708697 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ce46b-f8a6-4766-982e-6b93ce8f8422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:06.709384 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ce46b-f8a6-4766-982e-6b93ce8f8422 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:06.709672 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ce46b-f8a6-4766-982e-6b93ce8f8422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:06.709953 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ce46b-f8a6-4766-982e-6b93ce8f8422 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:06.710669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ce46b-f8a6-4766-982e-6b93ce8f8422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:06.712259 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ce46b-f8a6-4766-982e-6b93ce8f8422 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:06.723287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1410/2805] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 20:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 20:02:06.727279 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574796b-20b7-4d03-ace2-cc7beac4ddc6 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:06.737213 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9574796b-20b7-4d03-ace2-cc7beac4ddc6 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:06.741775 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb1102-ec72-47c7-b6fe-6e96b3735b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:06.742647 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb1102-ec72-47c7-b6fe-6e96b3735b00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:06.742931 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb1102-ec72-47c7-b6fe-6e96b3735b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:06.743206 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb1102-ec72-47c7-b6fe-6e96b3735b00 admin admin] PATH_INFO: `/v3/projects/7e3e40e2ccf7472ea770bfb6e67d58e3/users/f0f5acd00b8a4af2bfd78c740d2219d8/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:06.757666 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb1102-ec72-47c7-b6fe-6e96b3735b00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e3e40e2ccf7472ea770bfb6e67d58e3, user_id=f0f5acd00b8a4af2bfd78c740d2219d8, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:06.760278 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb1102-ec72-47c7-b6fe-6e96b3735b00 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:06.827261 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1396/2806] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 20:02:06 2022] PUT /identity/v3/projects/7e3e40e2ccf7472ea770bfb6e67d58e3/users/f0f5acd00b8a4af2bfd78c740d2219d8/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:02:06.831124 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ce46b-f8a6-4766-982e-6b93ce8f8422 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:06.841123 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ce46b-f8a6-4766-982e-6b93ce8f8422 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:06.896869 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093880cc-b0e1-4358-9e1e-3a3a3329b38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:06.897567 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093880cc-b0e1-4358-9e1e-3a3a3329b38e admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:06.897881 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093880cc-b0e1-4358-9e1e-3a3a3329b38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:06.898155 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093880cc-b0e1-4358-9e1e-3a3a3329b38e admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:06.898867 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093880cc-b0e1-4358-9e1e-3a3a3329b38e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:06.900488 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093880cc-b0e1-4358-9e1e-3a3a3329b38e admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:06.910954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1411/2807] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 20:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 20:02:06.915065 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb1102-ec72-47c7-b6fe-6e96b3735b00 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:06.925456 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb1102-ec72-47c7-b6fe-6e96b3735b00 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:06.930322 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff864c-d4b2-4b19-bf23-ddc6b0e23b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:06.931207 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff864c-d4b2-4b19-bf23-ddc6b0e23b01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:06.931493 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff864c-d4b2-4b19-bf23-ddc6b0e23b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:06.931771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff864c-d4b2-4b19-bf23-ddc6b0e23b01 admin admin] PATH_INFO: `/v3/projects/7e3e40e2ccf7472ea770bfb6e67d58e3/users/f0f5acd00b8a4af2bfd78c740d2219d8/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:06.933747 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff864c-d4b2-4b19-bf23-ddc6b0e23b01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e3e40e2ccf7472ea770bfb6e67d58e3, user_id=f0f5acd00b8a4af2bfd78c740d2219d8, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:06.936668 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff864c-d4b2-4b19-bf23-ddc6b0e23b01 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:06.946744 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1397/2808] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 20:02:06 2022] PUT /identity/v3/projects/7e3e40e2ccf7472ea770bfb6e67d58e3/users/f0f5acd00b8a4af2bfd78c740d2219d8/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:02:06.959013 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a22071-92da-4b42-8bc6-8e7657272ee4 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:06.959306 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a22071-92da-4b42-8bc6-8e7657272ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:06.959598 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a22071-92da-4b42-8bc6-8e7657272ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:06.985477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-e9a22071-92da-4b42-8bc6-8e7657272ee4 None None] MFA Rules not processed for user `f0f5acd00b8a4af2bfd78c740d2219d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:02:06.986679 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-e9a22071-92da-4b42-8bc6-8e7657272ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:07.052275 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1412/2809] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:02:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:02:07.069268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff864c-d4b2-4b19-bf23-ddc6b0e23b01 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:07.071003 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-d8ff864c-d4b2-4b19-bf23-ddc6b0e23b01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:07.102563 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:02:07.139878 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff864c-d4b2-4b19-bf23-ddc6b0e23b01 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:07.144125 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a4083-7902-40e1-ada5-571fb4c128b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:07.144293 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a4083-7902-40e1-ada5-571fb4c128b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:07.144445 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a4083-7902-40e1-ada5-571fb4c128b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:07.144721 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a4083-7902-40e1-ada5-571fb4c128b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:07.153684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a4083-7902-40e1-ada5-571fb4c128b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:07.155280 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a4083-7902-40e1-ada5-571fb4c128b4 service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:07.164718 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1398/2810] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:02:07 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:02:08.099555 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a22071-92da-4b42-8bc6-8e7657272ee4 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:08.109344 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a22071-92da-4b42-8bc6-8e7657272ee4 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:08.113931 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b7ed1-f516-49a5-8883-e7dc72e1cfd7 tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0f5acd00b8a4af2bfd78c740d2219d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e3e40e2ccf7472ea770bfb6e67d58e3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:08.114598 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b7ed1-f516-49a5-8883-e7dc72e1cfd7 tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:08.114889 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b7ed1-f516-49a5-8883-e7dc72e1cfd7 tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:08.115175 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b7ed1-f516-49a5-8883-e7dc72e1cfd7 tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:08.115907 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b7ed1-f516-49a5-8883-e7dc72e1cfd7 tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 20:02:08.116224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b7ed1-f516-49a5-8883-e7dc72e1cfd7 tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:08.117902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b7ed1-f516-49a5-8883-e7dc72e1cfd7 tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:08.129889 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1413/2811] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 20:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 20:02:08.133847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a4083-7902-40e1-ada5-571fb4c128b4 service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:08.142999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a4083-7902-40e1-ada5-571fb4c128b4 service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:08.147487 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e4c26-9849-4172-b2b3-3a36e72ce7da tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0f5acd00b8a4af2bfd78c740d2219d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e3e40e2ccf7472ea770bfb6e67d58e3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:08.148097 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e4c26-9849-4172-b2b3-3a36e72ce7da tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:08.148382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e4c26-9849-4172-b2b3-3a36e72ce7da tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:08.148659 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e4c26-9849-4172-b2b3-3a36e72ce7da tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:08.149389 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e4c26-9849-4172-b2b3-3a36e72ce7da tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:08.151334 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e4c26-9849-4172-b2b3-3a36e72ce7da tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:08.215449 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1399/2812] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 20:02:08 2022] POST /identity/v3/projects => generated 353 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 20:02:08.219553 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b7ed1-f516-49a5-8883-e7dc72e1cfd7 tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:08.229859 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b7ed1-f516-49a5-8883-e7dc72e1cfd7 tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:08.277243 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=100 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:02:08.285993 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66183fb3-a1c4-4031-a1f3-1018c4f9741c tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0f5acd00b8a4af2bfd78c740d2219d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e3e40e2ccf7472ea770bfb6e67d58e3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:08.286659 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66183fb3-a1c4-4031-a1f3-1018c4f9741c tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:08.286947 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66183fb3-a1c4-4031-a1f3-1018c4f9741c tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:08.287227 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66183fb3-a1c4-4031-a1f3-1018c4f9741c tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:08.287958 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66183fb3-a1c4-4031-a1f3-1018c4f9741c tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 20:02:08.288267 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66183fb3-a1c4-4031-a1f3-1018c4f9741c tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:08.289902 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66183fb3-a1c4-4031-a1f3-1018c4f9741c tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:08.302165 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1414/2813] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 20:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 20:02:08.339350 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e4c26-9849-4172-b2b3-3a36e72ce7da tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:08.349508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e4c26-9849-4172-b2b3-3a36e72ce7da tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:08.354076 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d2cc9-14de-43cc-bd2c-1c04f6af4b66 tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f0f5acd00b8a4af2bfd78c740d2219d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e3e40e2ccf7472ea770bfb6e67d58e3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:08.354932 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d2cc9-14de-43cc-bd2c-1c04f6af4b66 tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:08.355223 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d2cc9-14de-43cc-bd2c-1c04f6af4b66 tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:08.355503 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d2cc9-14de-43cc-bd2c-1c04f6af4b66 tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] PATH_INFO: `/v3/projects/4926b7cdcd7849508d12d72be4af0e83` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:08.356684 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d2cc9-14de-43cc-bd2c-1c04f6af4b66 tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] RBAC: Authorizing `identity:delete_project(project_id=4926b7cdcd7849508d12d72be4af0e83)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:08.359106 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d2cc9-14de-43cc-bd2c-1c04f6af4b66 tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:08.470341 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f08d2cc9-14de-43cc-bd2c-1c04f6af4b66 tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4926b7cdcd7849508d12d72be4af0e83', 'request_id': 'req-f08d2cc9-14de-43cc-bd2c-1c04f6af4b66'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:08.475943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f08d2cc9-14de-43cc-bd2c-1c04f6af4b66 tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4926b7cdcd7849508d12d72be4af0e83', 'request_id': 'req-f08d2cc9-14de-43cc-bd2c-1c04f6af4b66'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:08.544138 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f08d2cc9-14de-43cc-bd2c-1c04f6af4b66 tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4926b7cdcd7849508d12d72be4af0e83', 'request_id': 'req-f08d2cc9-14de-43cc-bd2c-1c04f6af4b66'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:08.557140 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f08d2cc9-14de-43cc-bd2c-1c04f6af4b66 tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] The token cache is being invalidate because project 4926b7cdcd7849508d12d72be4af0e83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 20:02:08.557435 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f08d2cc9-14de-43cc-bd2c-1c04f6af4b66 tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:08.558084 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-f08d2cc9-14de-43cc-bd2c-1c04f6af4b66 tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:08.559638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1400/2814] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 20:02:08 2022] DELETE /identity/v3/projects/4926b7cdcd7849508d12d72be4af0e83 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:02:09.127046 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66183fb3-a1c4-4031-a1f3-1018c4f9741c tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:09.128832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-66183fb3-a1c4-4031-a1f3-1018c4f9741c tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:09.193351 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66183fb3-a1c4-4031-a1f3-1018c4f9741c tempest-VolumeQuotaClassesTest-562194025 tempest-VolumeQuotaClassesTest-562194025-project-admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:09.248344 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cb155-6151-41f2-afa9-5760a078270c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:09.249331 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cb155-6151-41f2-afa9-5760a078270c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:09.249615 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cb155-6151-41f2-afa9-5760a078270c admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:09.249891 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0cb155-6151-41f2-afa9-5760a078270c admin admin] PATH_INFO: `/v3/users/8e8193cb9fb74e1aa235f5e54aec7edc` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:09.251069 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0cb155-6151-41f2-afa9-5760a078270c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e8193cb9fb74e1aa235f5e54aec7edc)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:09.253004 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0cb155-6151-41f2-afa9-5760a078270c admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:09.415786 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-7f0cb155-6151-41f2-afa9-5760a078270c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e8193cb9fb74e1aa235f5e54aec7edc'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:09.427545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-7f0cb155-6151-41f2-afa9-5760a078270c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e8193cb9fb74e1aa235f5e54aec7edc'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:09.488199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-7f0cb155-6151-41f2-afa9-5760a078270c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e8193cb9fb74e1aa235f5e54aec7edc'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:09.488825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-7f0cb155-6151-41f2-afa9-5760a078270c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e8193cb9fb74e1aa235f5e54aec7edc'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:09.496234 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-7f0cb155-6151-41f2-afa9-5760a078270c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e8193cb9fb74e1aa235f5e54aec7edc'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:09.502809 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1415/2815] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 20:02:09 2022] DELETE /identity/v3/users/8e8193cb9fb74e1aa235f5e54aec7edc => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:02:09.507282 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf370bdc-f532-4f20-ae89-a57130b50d98 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:09.507809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf370bdc-f532-4f20-ae89-a57130b50d98 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:09.507917 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf370bdc-f532-4f20-ae89-a57130b50d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:09.525302 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-bf370bdc-f532-4f20-ae89-a57130b50d98 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:02:09.526495 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-bf370bdc-f532-4f20-ae89-a57130b50d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:09.585352 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1401/2816] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:02:09.595024 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cb155-6151-41f2-afa9-5760a078270c admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:09.596410 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-7f0cb155-6151-41f2-afa9-5760a078270c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:09.611102 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:02:09.654028 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0cb155-6151-41f2-afa9-5760a078270c admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:09.657838 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598dd1c-c544-4a36-b788-8de6864bfcb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:09.658495 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598dd1c-c544-4a36-b788-8de6864bfcb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:09.658954 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598dd1c-c544-4a36-b788-8de6864bfcb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:09.659244 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5598dd1c-c544-4a36-b788-8de6864bfcb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:09.670451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598dd1c-c544-4a36-b788-8de6864bfcb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:09.671965 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5598dd1c-c544-4a36-b788-8de6864bfcb4 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:09.681493 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1416/2817] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:02:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:02:10.039032 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf370bdc-f532-4f20-ae89-a57130b50d98 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:10.040190 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-bf370bdc-f532-4f20-ae89-a57130b50d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:10.053403 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf370bdc-f532-4f20-ae89-a57130b50d98 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:10.058012 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac786829-7699-48fe-8741-2e0a0d7b2e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:10.058882 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac786829-7699-48fe-8741-2e0a0d7b2e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:10.059170 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac786829-7699-48fe-8741-2e0a0d7b2e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:10.059449 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac786829-7699-48fe-8741-2e0a0d7b2e3e admin admin] PATH_INFO: `/v3/projects/69730ec093f74ebaaebc8c6980ff6f6b` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:10.060677 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac786829-7699-48fe-8741-2e0a0d7b2e3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=69730ec093f74ebaaebc8c6980ff6f6b)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:10.062627 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac786829-7699-48fe-8741-2e0a0d7b2e3e admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:10.079748 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:02:10.221014 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-ac786829-7699-48fe-8741-2e0a0d7b2e3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69730ec093f74ebaaebc8c6980ff6f6b', 'request_id': 'req-ac786829-7699-48fe-8741-2e0a0d7b2e3e'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:10.226735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-ac786829-7699-48fe-8741-2e0a0d7b2e3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69730ec093f74ebaaebc8c6980ff6f6b', 'request_id': 'req-ac786829-7699-48fe-8741-2e0a0d7b2e3e'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:10.317047 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-ac786829-7699-48fe-8741-2e0a0d7b2e3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69730ec093f74ebaaebc8c6980ff6f6b', 'request_id': 'req-ac786829-7699-48fe-8741-2e0a0d7b2e3e'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:10.329093 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-ac786829-7699-48fe-8741-2e0a0d7b2e3e admin admin] The token cache is being invalidate because project 69730ec093f74ebaaebc8c6980ff6f6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 20:02:10.329404 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-ac786829-7699-48fe-8741-2e0a0d7b2e3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:10.330025 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-ac786829-7699-48fe-8741-2e0a0d7b2e3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:10.331550 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1402/2818] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 20:02:10 2022] DELETE /identity/v3/projects/69730ec093f74ebaaebc8c6980ff6f6b => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:02:10.335606 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598dd1c-c544-4a36-b788-8de6864bfcb4 service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:10.345682 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5598dd1c-c544-4a36-b788-8de6864bfcb4 service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:10.400616 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ff4bce-9992-4ae6-b16a-98f836a657e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:10.401512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff4bce-9992-4ae6-b16a-98f836a657e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:10.401806 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff4bce-9992-4ae6-b16a-98f836a657e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:10.402081 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ff4bce-9992-4ae6-b16a-98f836a657e7 admin admin] PATH_INFO: `/v3/users/f0f5acd00b8a4af2bfd78c740d2219d8` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:10.403248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff4bce-9992-4ae6-b16a-98f836a657e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0f5acd00b8a4af2bfd78c740d2219d8)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:10.405956 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ff4bce-9992-4ae6-b16a-98f836a657e7 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:10.584059 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-08ff4bce-9992-4ae6-b16a-98f836a657e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0f5acd00b8a4af2bfd78c740d2219d8'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:10.596744 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-08ff4bce-9992-4ae6-b16a-98f836a657e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0f5acd00b8a4af2bfd78c740d2219d8'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:10.654917 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-08ff4bce-9992-4ae6-b16a-98f836a657e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0f5acd00b8a4af2bfd78c740d2219d8'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:10.655545 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-08ff4bce-9992-4ae6-b16a-98f836a657e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0f5acd00b8a4af2bfd78c740d2219d8'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:10.662683 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-08ff4bce-9992-4ae6-b16a-98f836a657e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0f5acd00b8a4af2bfd78c740d2219d8'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:10.668785 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1417/2819] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 20:02:10 2022] DELETE /identity/v3/users/f0f5acd00b8a4af2bfd78c740d2219d8 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:02:11.071675 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac786829-7699-48fe-8741-2e0a0d7b2e3e admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:11.073224 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-ac786829-7699-48fe-8741-2e0a0d7b2e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:11.087305 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac786829-7699-48fe-8741-2e0a0d7b2e3e admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:11.144851 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c4c45b-9c46-40ac-9628-c6201005e321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:11.145682 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4c45b-9c46-40ac-9628-c6201005e321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:11.145968 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4c45b-9c46-40ac-9628-c6201005e321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:11.146246 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c4c45b-9c46-40ac-9628-c6201005e321 admin admin] PATH_INFO: `/v3/projects/7e3e40e2ccf7472ea770bfb6e67d58e3` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:11.147426 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c4c45b-9c46-40ac-9628-c6201005e321 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e3e40e2ccf7472ea770bfb6e67d58e3)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:11.149365 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c4c45b-9c46-40ac-9628-c6201005e321 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:11.269423 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-15c4c45b-9c46-40ac-9628-c6201005e321 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e3e40e2ccf7472ea770bfb6e67d58e3', 'request_id': 'req-15c4c45b-9c46-40ac-9628-c6201005e321'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:11.274964 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-15c4c45b-9c46-40ac-9628-c6201005e321 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e3e40e2ccf7472ea770bfb6e67d58e3', 'request_id': 'req-15c4c45b-9c46-40ac-9628-c6201005e321'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:11.334163 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-15c4c45b-9c46-40ac-9628-c6201005e321 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e3e40e2ccf7472ea770bfb6e67d58e3', 'request_id': 'req-15c4c45b-9c46-40ac-9628-c6201005e321'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:11.345494 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-15c4c45b-9c46-40ac-9628-c6201005e321 admin admin] The token cache is being invalidate because project 7e3e40e2ccf7472ea770bfb6e67d58e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 20:02:11.345781 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-15c4c45b-9c46-40ac-9628-c6201005e321 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:11.346404 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-15c4c45b-9c46-40ac-9628-c6201005e321 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:11.347933 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1403/2820] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 20:02:11 2022] DELETE /identity/v3/projects/7e3e40e2ccf7472ea770bfb6e67d58e3 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:02:11.353660 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1d241-948f-42a0-b36d-769e0cd43cf0 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:11.354126 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1d241-948f-42a0-b36d-769e0cd43cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:11.354224 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1d241-948f-42a0-b36d-769e0cd43cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:11.373442 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-94a1d241-948f-42a0-b36d-769e0cd43cf0 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:02:11.374644 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-94a1d241-948f-42a0-b36d-769e0cd43cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:11.403258 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:02:11.431850 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1418/2821] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:02:11.439968 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8742502-17a9-4d4b-86ce-a93d80986590 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:11.444009 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8742502-17a9-4d4b-86ce-a93d80986590 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:11.444009 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8742502-17a9-4d4b-86ce-a93d80986590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:11.458859 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-a8742502-17a9-4d4b-86ce-a93d80986590 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:02:11.460051 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-a8742502-17a9-4d4b-86ce-a93d80986590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:11.468097 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1404/2822] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:02:11.472199 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1d241-948f-42a0-b36d-769e0cd43cf0 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:11.482244 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1d241-948f-42a0-b36d-769e0cd43cf0 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:11.486972 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6b326-0818-47c3-b0a5-806ca75d650c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:11.487842 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6b326-0818-47c3-b0a5-806ca75d650c admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:11.488317 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6b326-0818-47c3-b0a5-806ca75d650c admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:11.488631 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6b326-0818-47c3-b0a5-806ca75d650c admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:11.489662 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6b326-0818-47c3-b0a5-806ca75d650c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 20:02:11.489974 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6b326-0818-47c3-b0a5-806ca75d650c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:11.491590 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6b326-0818-47c3-b0a5-806ca75d650c admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:11.503714 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1419/2823] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 20:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 20:02:11.508012 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8742502-17a9-4d4b-86ce-a93d80986590 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:11.517478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8742502-17a9-4d4b-86ce-a93d80986590 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:11.522142 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02788a91-713a-4713-9fe7-2f9f3f03b3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:11.522803 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02788a91-713a-4713-9fe7-2f9f3f03b3d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:11.523091 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02788a91-713a-4713-9fe7-2f9f3f03b3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:11.523371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02788a91-713a-4713-9fe7-2f9f3f03b3d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:11.524098 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02788a91-713a-4713-9fe7-2f9f3f03b3d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:11.526040 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02788a91-713a-4713-9fe7-2f9f3f03b3d0 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:11.596993 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1405/2824] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 20:02:11 2022] POST /identity/v3/projects => generated 373 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 20:02:11.601109 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6b326-0818-47c3-b0a5-806ca75d650c admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:11.611741 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6b326-0818-47c3-b0a5-806ca75d650c admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:11.666037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e7404-e774-4bab-9518-58b81aa879b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:11.666628 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e7404-e774-4bab-9518-58b81aa879b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:11.666911 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e7404-e774-4bab-9518-58b81aa879b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:11.667188 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209e7404-e774-4bab-9518-58b81aa879b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:11.667868 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e7404-e774-4bab-9518-58b81aa879b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:11.670399 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209e7404-e774-4bab-9518-58b81aa879b4 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:11.738353 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1420/2825] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 20:02:11 2022] POST /identity/v3/users => generated 287 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 20:02:11.742439 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02788a91-713a-4713-9fe7-2f9f3f03b3d0 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:11.752847 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02788a91-713a-4713-9fe7-2f9f3f03b3d0 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:11.757523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b1a7e-41b5-43a6-8b6c-387c1f147341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:11.758182 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b1a7e-41b5-43a6-8b6c-387c1f147341 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:11.758466 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b1a7e-41b5-43a6-8b6c-387c1f147341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:11.758752 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b1a7e-41b5-43a6-8b6c-387c1f147341 admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:11.759479 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b1a7e-41b5-43a6-8b6c-387c1f147341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:11.761106 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b1a7e-41b5-43a6-8b6c-387c1f147341 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:11.762336 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:02:11.773052 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1406/2826] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 20:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 20:02:11.776994 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e7404-e774-4bab-9518-58b81aa879b4 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:11.787384 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209e7404-e774-4bab-9518-58b81aa879b4 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:11.791905 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a4b9e4-7d4e-458e-8d2b-dad6b2e11b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:11.792557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4b9e4-7d4e-458e-8d2b-dad6b2e11b4b admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:11.792842 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4b9e4-7d4e-458e-8d2b-dad6b2e11b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:11.793187 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4b9e4-7d4e-458e-8d2b-dad6b2e11b4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:11.793936 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a4b9e4-7d4e-458e-8d2b-dad6b2e11b4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:11.796299 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a4b9e4-7d4e-458e-8d2b-dad6b2e11b4b admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:11.806406 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1421/2827] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 20:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 20:02:11.810458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b1a7e-41b5-43a6-8b6c-387c1f147341 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:11.820736 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b1a7e-41b5-43a6-8b6c-387c1f147341 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:11.825759 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebcba4-d913-4223-af69-735f7518bd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:11.826960 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebcba4-d913-4223-af69-735f7518bd86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:11.827126 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebcba4-d913-4223-af69-735f7518bd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:11.827283 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebcba4-d913-4223-af69-735f7518bd86 admin admin] PATH_INFO: `/v3/projects/1fcfc61bdd7846608539bbab995914b8/users/68b2cfd5629d4915b11ebad8ee02dad0/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:11.843426 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebcba4-d913-4223-af69-735f7518bd86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fcfc61bdd7846608539bbab995914b8, user_id=68b2cfd5629d4915b11ebad8ee02dad0, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:11.846109 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebcba4-d913-4223-af69-735f7518bd86 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:11.894742 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1407/2828] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 20:02:11 2022] PUT /identity/v3/projects/1fcfc61bdd7846608539bbab995914b8/users/68b2cfd5629d4915b11ebad8ee02dad0/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:02:11.910241 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fcd98-8e98-4e74-a763-de176bc5077d None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:11.910534 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fcd98-8e98-4e74-a763-de176bc5077d None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:11.910817 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4fcd98-8e98-4e74-a763-de176bc5077d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:11.937225 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-ba4fcd98-8e98-4e74-a763-de176bc5077d None None] MFA Rules not processed for user `68b2cfd5629d4915b11ebad8ee02dad0`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:02:11.938355 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ba4fcd98-8e98-4e74-a763-de176bc5077d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:12.000211 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1422/2829] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:02:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:02:12.010614 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebcba4-d913-4223-af69-735f7518bd86 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:12.012286 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-7aebcba4-d913-4223-af69-735f7518bd86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:12.080732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebcba4-d913-4223-af69-735f7518bd86 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:12.084378 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf48b2c-d8d7-434b-a5f0-c6ca6c0157a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:12.085050 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf48b2c-d8d7-434b-a5f0-c6ca6c0157a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:12.085338 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf48b2c-d8d7-434b-a5f0-c6ca6c0157a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:12.085616 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf48b2c-d8d7-434b-a5f0-c6ca6c0157a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:12.094008 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf48b2c-d8d7-434b-a5f0-c6ca6c0157a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:12.095429 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf48b2c-d8d7-434b-a5f0-c6ca6c0157a7 service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:12.104156 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1408/2830] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:02:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:02:21.997248 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=83,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:02:22.100085 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=43,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:02:23.180746 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4fcd98-8e98-4e74-a763-de176bc5077d None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:23.241152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4fcd98-8e98-4e74-a763-de176bc5077d None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:23.245675 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce8000-02e8-4201-9e92-69a240adb9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:23.246527 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce8000-02e8-4201-9e92-69a240adb9ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:23.246808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce8000-02e8-4201-9e92-69a240adb9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:23.247082 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce8000-02e8-4201-9e92-69a240adb9ad admin admin] PATH_INFO: `/v3/users/68b2cfd5629d4915b11ebad8ee02dad0` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:23.248551 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce8000-02e8-4201-9e92-69a240adb9ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=68b2cfd5629d4915b11ebad8ee02dad0)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:23.250509 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce8000-02e8-4201-9e92-69a240adb9ad admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:23.420662 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-b5ce8000-02e8-4201-9e92-69a240adb9ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68b2cfd5629d4915b11ebad8ee02dad0'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:23.432210 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-b5ce8000-02e8-4201-9e92-69a240adb9ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68b2cfd5629d4915b11ebad8ee02dad0'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:23.497368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-b5ce8000-02e8-4201-9e92-69a240adb9ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68b2cfd5629d4915b11ebad8ee02dad0'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:23.498523 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-b5ce8000-02e8-4201-9e92-69a240adb9ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68b2cfd5629d4915b11ebad8ee02dad0'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:23.505939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-b5ce8000-02e8-4201-9e92-69a240adb9ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68b2cfd5629d4915b11ebad8ee02dad0'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:23.512472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1423/2831] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 20:02:23 2022] DELETE /identity/v3/users/68b2cfd5629d4915b11ebad8ee02dad0 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:02:23.517220 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e3c0a-3555-4a23-9597-49ff08dc3e02 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:23.517683 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e3c0a-3555-4a23-9597-49ff08dc3e02 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:23.517908 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e3c0a-3555-4a23-9597-49ff08dc3e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:23.535720 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-6c8e3c0a-3555-4a23-9597-49ff08dc3e02 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:02:23.536936 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-6c8e3c0a-3555-4a23-9597-49ff08dc3e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:23.594482 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1409/2832] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:02:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:02:23.604087 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce8000-02e8-4201-9e92-69a240adb9ad admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:23.605504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-b5ce8000-02e8-4201-9e92-69a240adb9ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:23.660826 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce8000-02e8-4201-9e92-69a240adb9ad admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:23.664460 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc63a2c-cee9-4130-a54e-1a7fffb6ab2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:23.665140 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc63a2c-cee9-4130-a54e-1a7fffb6ab2a service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:23.665423 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc63a2c-cee9-4130-a54e-1a7fffb6ab2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:23.665697 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc63a2c-cee9-4130-a54e-1a7fffb6ab2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:23.675637 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc63a2c-cee9-4130-a54e-1a7fffb6ab2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:23.677035 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc63a2c-cee9-4130-a54e-1a7fffb6ab2a service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:23.686973 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1424/2833] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:02:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:02:24.071734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e3c0a-3555-4a23-9597-49ff08dc3e02 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:24.073206 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-6c8e3c0a-3555-4a23-9597-49ff08dc3e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:24.086364 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e3c0a-3555-4a23-9597-49ff08dc3e02 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:24.090978 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1a661a-4001-4784-a863-7cf4e4202bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:24.091891 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a661a-4001-4784-a863-7cf4e4202bb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:24.092170 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a661a-4001-4784-a863-7cf4e4202bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:24.092444 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1a661a-4001-4784-a863-7cf4e4202bb3 admin admin] PATH_INFO: `/v3/projects/1fcfc61bdd7846608539bbab995914b8` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:24.093657 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1a661a-4001-4784-a863-7cf4e4202bb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fcfc61bdd7846608539bbab995914b8)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:24.095679 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1a661a-4001-4784-a863-7cf4e4202bb3 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:24.179175 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-bb1a661a-4001-4784-a863-7cf4e4202bb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fcfc61bdd7846608539bbab995914b8', 'request_id': 'req-bb1a661a-4001-4784-a863-7cf4e4202bb3'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:24.184821 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-bb1a661a-4001-4784-a863-7cf4e4202bb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fcfc61bdd7846608539bbab995914b8', 'request_id': 'req-bb1a661a-4001-4784-a863-7cf4e4202bb3'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:24.241784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-bb1a661a-4001-4784-a863-7cf4e4202bb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fcfc61bdd7846608539bbab995914b8', 'request_id': 'req-bb1a661a-4001-4784-a863-7cf4e4202bb3'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:24.253088 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-bb1a661a-4001-4784-a863-7cf4e4202bb3 admin admin] The token cache is being invalidate because project 1fcfc61bdd7846608539bbab995914b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 20:02:24.253381 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-bb1a661a-4001-4784-a863-7cf4e4202bb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:24.254016 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-bb1a661a-4001-4784-a863-7cf4e4202bb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:02:24.255572 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1410/2834] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 20:02:24 2022] DELETE /identity/v3/projects/1fcfc61bdd7846608539bbab995914b8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:02:24.260449 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754102c3-d0d4-43d1-a379-b9696e3b7bd7 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:24.260934 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754102c3-d0d4-43d1-a379-b9696e3b7bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:24.261056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754102c3-d0d4-43d1-a379-b9696e3b7bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:24.279367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-754102c3-d0d4-43d1-a379-b9696e3b7bd7 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:02:24.280649 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-754102c3-d0d4-43d1-a379-b9696e3b7bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:24.298016 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:02:24.338684 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1425/2835] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:02:24.346866 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93775178-f2d2-4241-8094-9e7268069ce9 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:24.347166 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93775178-f2d2-4241-8094-9e7268069ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:24.347438 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93775178-f2d2-4241-8094-9e7268069ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:24.366268 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-93775178-f2d2-4241-8094-9e7268069ce9 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:02:24.370687 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-93775178-f2d2-4241-8094-9e7268069ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:24.376012 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1411/2836] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:02:24.380152 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754102c3-d0d4-43d1-a379-b9696e3b7bd7 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:24.390445 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754102c3-d0d4-43d1-a379-b9696e3b7bd7 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:24.395603 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7ee22-15fe-436f-9a26-99ecbfe45611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:24.396255 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7ee22-15fe-436f-9a26-99ecbfe45611 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:24.396541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7ee22-15fe-436f-9a26-99ecbfe45611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:24.396822 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7ee22-15fe-436f-9a26-99ecbfe45611 admin admin] PATH_INFO: `/v3/domains` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:24.397593 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7ee22-15fe-436f-9a26-99ecbfe45611 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80635) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 20:02:24.397929 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7ee22-15fe-436f-9a26-99ecbfe45611 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:24.399536 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a7ee22-15fe-436f-9a26-99ecbfe45611 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:24.411016 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1426/2837] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 20:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 20:02:24.415266 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93775178-f2d2-4241-8094-9e7268069ce9 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:24.424654 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93775178-f2d2-4241-8094-9e7268069ce9 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:24.429368 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33466046-f3b4-432f-9c94-cbd18a7bd1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:24.430016 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33466046-f3b4-432f-9c94-cbd18a7bd1f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:24.430299 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33466046-f3b4-432f-9c94-cbd18a7bd1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:24.430577 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33466046-f3b4-432f-9c94-cbd18a7bd1f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:24.431264 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33466046-f3b4-432f-9c94-cbd18a7bd1f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:24.433173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33466046-f3b4-432f-9c94-cbd18a7bd1f5 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:24.512120 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1412/2838] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 20:02:24 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 20:02:24.517061 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7ee22-15fe-436f-9a26-99ecbfe45611 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:24.527000 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a7ee22-15fe-436f-9a26-99ecbfe45611 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:24.582087 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9152f71-3d6a-460e-9fde-d774922989ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:24.582750 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9152f71-3d6a-460e-9fde-d774922989ff admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:24.583031 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9152f71-3d6a-460e-9fde-d774922989ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:24.583305 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9152f71-3d6a-460e-9fde-d774922989ff admin admin] PATH_INFO: `/v3/users` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:24.583989 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9152f71-3d6a-460e-9fde-d774922989ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:24.585943 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9152f71-3d6a-460e-9fde-d774922989ff admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:24.653301 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1427/2839] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 20:02:24 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 20:02:24.657683 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33466046-f3b4-432f-9c94-cbd18a7bd1f5 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:24.667781 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33466046-f3b4-432f-9c94-cbd18a7bd1f5 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:24.673064 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a55cc-cf33-42f4-9cbf-e94f621ed81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:24.673719 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a55cc-cf33-42f4-9cbf-e94f621ed81d admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:24.674002 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a55cc-cf33-42f4-9cbf-e94f621ed81d admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:24.674279 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a55cc-cf33-42f4-9cbf-e94f621ed81d admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:24.675057 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a55cc-cf33-42f4-9cbf-e94f621ed81d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:24.676629 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a55cc-cf33-42f4-9cbf-e94f621ed81d admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:24.688135 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1413/2840] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 20:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 20:02:24.692139 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9152f71-3d6a-460e-9fde-d774922989ff admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:24.702610 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9152f71-3d6a-460e-9fde-d774922989ff admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:24.707200 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936cfef3-91cb-4c4f-af47-98fdce9ad338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:24.707847 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cfef3-91cb-4c4f-af47-98fdce9ad338 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:24.708129 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cfef3-91cb-4c4f-af47-98fdce9ad338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:24.708406 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936cfef3-91cb-4c4f-af47-98fdce9ad338 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:24.709162 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936cfef3-91cb-4c4f-af47-98fdce9ad338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:24.710748 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936cfef3-91cb-4c4f-af47-98fdce9ad338 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:24.720540 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1428/2841] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 20:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 20:02:24.724515 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a55cc-cf33-42f4-9cbf-e94f621ed81d admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:24.734853 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a55cc-cf33-42f4-9cbf-e94f621ed81d admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:24.739467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb293070-17f0-4449-b738-365d449371b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:24.740348 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb293070-17f0-4449-b738-365d449371b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:24.740633 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb293070-17f0-4449-b738-365d449371b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:24.740909 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb293070-17f0-4449-b738-365d449371b6 admin admin] PATH_INFO: `/v3/projects/0899ff6998fc48d6a83240edb7f27a70/users/d6561b445d3748b8ab088468d9ffe019/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:24.755708 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb293070-17f0-4449-b738-365d449371b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0899ff6998fc48d6a83240edb7f27a70, user_id=d6561b445d3748b8ab088468d9ffe019, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:24.758406 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb293070-17f0-4449-b738-365d449371b6 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:24.870903 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1414/2842] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 20:02:24 2022] PUT /identity/v3/projects/0899ff6998fc48d6a83240edb7f27a70/users/d6561b445d3748b8ab088468d9ffe019/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:02:24.876612 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d3286-a789-41b0-9d21-761a3002c14a None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:24.876908 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d3286-a789-41b0-9d21-761a3002c14a None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:24.877216 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d3286-a789-41b0-9d21-761a3002c14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:24.895774 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-c14d3286-a789-41b0-9d21-761a3002c14a None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:02:24.897009 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-c14d3286-a789-41b0-9d21-761a3002c14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:24.955137 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1429/2843] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:02:24.964595 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb293070-17f0-4449-b738-365d449371b6 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:24.966780 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-fb293070-17f0-4449-b738-365d449371b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:24.988507 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:02:25.023815 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb293070-17f0-4449-b738-365d449371b6 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:25.027577 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5cd88-6af5-46a7-a174-5b9ea052adda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:25.028221 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5cd88-6af5-46a7-a174-5b9ea052adda service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:25.028509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5cd88-6af5-46a7-a174-5b9ea052adda service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:25.029342 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e5cd88-6af5-46a7-a174-5b9ea052adda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:25.038451 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5cd88-6af5-46a7-a174-5b9ea052adda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:25.039878 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e5cd88-6af5-46a7-a174-5b9ea052adda service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:25.050539 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1415/2844] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:02:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:02:26.334557 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15c026-54b4-4ced-b029-53714f860f74 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:26.335173 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15c026-54b4-4ced-b029-53714f860f74 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:26.335173 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15c026-54b4-4ced-b029-53714f860f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:26.362537 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-5d15c026-54b4-4ced-b029-53714f860f74 None None] MFA Rules not processed for user `d6561b445d3748b8ab088468d9ffe019`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:02:26.363738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-5d15c026-54b4-4ced-b029-53714f860f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:26.382194 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:02:26.431449 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1430/2845] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:02:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:02:26.442865 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5cd88-6af5-46a7-a174-5b9ea052adda service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:26.444292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-b3e5cd88-6af5-46a7-a174-5b9ea052adda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:26.501197 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e5cd88-6af5-46a7-a174-5b9ea052adda service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:26.504849 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49518f90-4c87-475e-a093-0b14e599a4ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:26.505531 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49518f90-4c87-475e-a093-0b14e599a4ae service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:26.505819 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49518f90-4c87-475e-a093-0b14e599a4ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:26.506102 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49518f90-4c87-475e-a093-0b14e599a4ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:26.515167 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49518f90-4c87-475e-a093-0b14e599a4ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:26.516613 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49518f90-4c87-475e-a093-0b14e599a4ae service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:26.525599 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1416/2846] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:02:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:02:28.150676 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbea846-8126-4c37-8246-cc57a4694dd3 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:28.151215 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbea846-8126-4c37-8246-cc57a4694dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:28.151215 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbea846-8126-4c37-8246-cc57a4694dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:28.169235 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-8dbea846-8126-4c37-8246-cc57a4694dd3 None None] MFA Rules not processed for user `d6561b445d3748b8ab088468d9ffe019`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:02:28.171341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-8dbea846-8126-4c37-8246-cc57a4694dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:28.179351 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1431/2847] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:02:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:02:36.521471 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=43 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:02:38.164208 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=22 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:02:38.212278 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49518f90-4c87-475e-a093-0b14e599a4ae service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:02:38.213809 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-49518f90-4c87-475e-a093-0b14e599a4ae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:38.294641 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49518f90-4c87-475e-a093-0b14e599a4ae service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:02:38.300391 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f10494-8e75-44eb-8d8a-b12e492d3153 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:02:38.301112 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f10494-8e75-44eb-8d8a-b12e492d3153 service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:02:38.301404 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f10494-8e75-44eb-8d8a-b12e492d3153 service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:02:38.301709 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f10494-8e75-44eb-8d8a-b12e492d3153 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:02:38.303562 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-63f10494-8e75-44eb-8d8a-b12e492d3153 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:02:38.316928 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f10494-8e75-44eb-8d8a-b12e492d3153 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:02:38.318617 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f10494-8e75-44eb-8d8a-b12e492d3153 service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:02:38.331021 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1417/2848] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:02:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:02:48.325947 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=37 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:03:34.615316 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64848c0-c433-4955-bf65-c2c37769ee41 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:03:34.615882 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64848c0-c433-4955-bf65-c2c37769ee41 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:03:34.616216 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64848c0-c433-4955-bf65-c2c37769ee41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:03:34.634556 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-a64848c0-c433-4955-bf65-c2c37769ee41 None None] MFA Rules not processed for user `d6561b445d3748b8ab088468d9ffe019`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:03:34.635854 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-a64848c0-c433-4955-bf65-c2c37769ee41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:03:34.648884 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1432/2849] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:03:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:03:44.628345 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:04:16.264116 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f10494-8e75-44eb-8d8a-b12e492d3153 service nova] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:04:16.265693 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-63f10494-8e75-44eb-8d8a-b12e492d3153 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:04:16.364508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f10494-8e75-44eb-8d8a-b12e492d3153 service nova] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:04:16.369523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e66b6b-d30e-4eb0-b5b5-d024fa8e8fba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2ed13a0c84f128b795002892efc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:04:16.370166 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e66b6b-d30e-4eb0-b5b5-d024fa8e8fba service placement] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:04:16.370450 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e66b6b-d30e-4eb0-b5b5-d024fa8e8fba service placement] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:04:16.370752 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e66b6b-d30e-4eb0-b5b5-d024fa8e8fba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:04:16.372503 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-20e66b6b-d30e-4eb0-b5b5-d024fa8e8fba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:04:16.383982 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e66b6b-d30e-4eb0-b5b5-d024fa8e8fba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:04:16.385597 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e66b6b-d30e-4eb0-b5b5-d024fa8e8fba service placement] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:04:16.396430 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1418/2850] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:04:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:04:26.391159 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=41 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:04:42.402949 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64848c0-c433-4955-bf65-c2c37769ee41 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:04:42.415292 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64848c0-c433-4955-bf65-c2c37769ee41 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:04:42.420808 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc771ac-cf80-48a0-b00e-6e62a1aa9f6c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2ed13a0c84f128b795002892efc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:04:42.421488 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc771ac-cf80-48a0-b00e-6e62a1aa9f6c service placement] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:04:42.421817 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc771ac-cf80-48a0-b00e-6e62a1aa9f6c service placement] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:04:42.422081 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc771ac-cf80-48a0-b00e-6e62a1aa9f6c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:04:42.423893 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-2cc771ac-cf80-48a0-b00e-6e62a1aa9f6c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:04:42.435835 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc771ac-cf80-48a0-b00e-6e62a1aa9f6c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:04:42.439917 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc771ac-cf80-48a0-b00e-6e62a1aa9f6c service placement] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:04:42.448434 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1433/2851] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:04:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:04:42.524833 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e66b6b-d30e-4eb0-b5b5-d024fa8e8fba service placement] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:04:42.536539 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e66b6b-d30e-4eb0-b5b5-d024fa8e8fba service placement] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:04:42.541765 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b05121e-a31d-4720-88fc-b5b17339f6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:04:42.542672 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05121e-a31d-4720-88fc-b5b17339f6c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:04:42.542954 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05121e-a31d-4720-88fc-b5b17339f6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:04:42.543228 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b05121e-a31d-4720-88fc-b5b17339f6c2 admin admin] PATH_INFO: `/v3/users/d6561b445d3748b8ab088468d9ffe019` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:04:42.544409 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05121e-a31d-4720-88fc-b5b17339f6c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6561b445d3748b8ab088468d9ffe019)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:04:42.546374 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b05121e-a31d-4720-88fc-b5b17339f6c2 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:04:42.700086 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-9b05121e-a31d-4720-88fc-b5b17339f6c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6561b445d3748b8ab088468d9ffe019'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:04:42.712225 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-9b05121e-a31d-4720-88fc-b5b17339f6c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6561b445d3748b8ab088468d9ffe019'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:04:42.794776 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-9b05121e-a31d-4720-88fc-b5b17339f6c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6561b445d3748b8ab088468d9ffe019'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:04:42.799714 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-9b05121e-a31d-4720-88fc-b5b17339f6c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6561b445d3748b8ab088468d9ffe019'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:04:42.799714 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-9b05121e-a31d-4720-88fc-b5b17339f6c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6561b445d3748b8ab088468d9ffe019'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:04:42.809122 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1419/2852] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 20:04:42 2022] DELETE /identity/v3/users/d6561b445d3748b8ab088468d9ffe019 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:04:43.311230 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc771ac-cf80-48a0-b00e-6e62a1aa9f6c service placement] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:04:43.321712 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc771ac-cf80-48a0-b00e-6e62a1aa9f6c service placement] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:04:43.384438 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f11f9-ea40-4e34-98e8-53cac6289886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:04:43.385947 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f11f9-ea40-4e34-98e8-53cac6289886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:04:43.386229 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f11f9-ea40-4e34-98e8-53cac6289886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:04:43.386504 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f11f9-ea40-4e34-98e8-53cac6289886 admin admin] PATH_INFO: `/v3/projects/0899ff6998fc48d6a83240edb7f27a70` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:04:43.387682 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f11f9-ea40-4e34-98e8-53cac6289886 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0899ff6998fc48d6a83240edb7f27a70)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:04:43.389635 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f11f9-ea40-4e34-98e8-53cac6289886 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:04:43.474488 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-b63f11f9-ea40-4e34-98e8-53cac6289886 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0899ff6998fc48d6a83240edb7f27a70', 'request_id': 'req-b63f11f9-ea40-4e34-98e8-53cac6289886'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:04:43.480006 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-b63f11f9-ea40-4e34-98e8-53cac6289886 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0899ff6998fc48d6a83240edb7f27a70', 'request_id': 'req-b63f11f9-ea40-4e34-98e8-53cac6289886'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:04:43.485939 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-b63f11f9-ea40-4e34-98e8-53cac6289886 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0899ff6998fc48d6a83240edb7f27a70', 'request_id': 'req-b63f11f9-ea40-4e34-98e8-53cac6289886'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:04:43.554888 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-b63f11f9-ea40-4e34-98e8-53cac6289886 admin admin] The token cache is being invalidate because project 0899ff6998fc48d6a83240edb7f27a70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 20:04:43.555101 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-b63f11f9-ea40-4e34-98e8-53cac6289886 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:04:43.555730 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-b63f11f9-ea40-4e34-98e8-53cac6289886 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:04:43.557829 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1434/2853] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 20:04:43 2022] DELETE /identity/v3/projects/0899ff6998fc48d6a83240edb7f27a70 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:04:43.562861 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf5fef-8b74-451d-9c0e-cf9a1d81a71d None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:04:43.563333 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf5fef-8b74-451d-9c0e-cf9a1d81a71d None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:04:43.563482 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf5fef-8b74-451d-9c0e-cf9a1d81a71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:04:43.581858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-91bf5fef-8b74-451d-9c0e-cf9a1d81a71d None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:04:43.583063 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-91bf5fef-8b74-451d-9c0e-cf9a1d81a71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:04:43.642606 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1420/2854] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:04:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:04:43.650408 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619ed9e-02d3-42a6-a0d6-0955800b5e34 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:04:43.650776 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619ed9e-02d3-42a6-a0d6-0955800b5e34 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:04:43.651058 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a619ed9e-02d3-42a6-a0d6-0955800b5e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:04:43.669386 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-a619ed9e-02d3-42a6-a0d6-0955800b5e34 None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:04:43.670580 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-a619ed9e-02d3-42a6-a0d6-0955800b5e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:04:43.678469 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1435/2855] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:04:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:04:43.682697 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf5fef-8b74-451d-9c0e-cf9a1d81a71d None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:04:43.692807 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf5fef-8b74-451d-9c0e-cf9a1d81a71d None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:04:43.697593 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddc79e-d8aa-49d9-b994-e87767b8ca9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:04:43.698252 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddc79e-d8aa-49d9-b994-e87767b8ca9b admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:04:43.698538 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddc79e-d8aa-49d9-b994-e87767b8ca9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:04:43.698816 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddc79e-d8aa-49d9-b994-e87767b8ca9b admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:04:43.699551 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ddc79e-d8aa-49d9-b994-e87767b8ca9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 20:04:43.699860 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ddc79e-d8aa-49d9-b994-e87767b8ca9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:04:43.701495 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ddc79e-d8aa-49d9-b994-e87767b8ca9b admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:04:43.713229 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1421/2856] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 20:04:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 20:04:43.717598 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619ed9e-02d3-42a6-a0d6-0955800b5e34 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:04:43.726686 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a619ed9e-02d3-42a6-a0d6-0955800b5e34 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:04:43.735547 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed1444-f2ad-4a4e-910e-df83a0347a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:04:43.736206 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed1444-f2ad-4a4e-910e-df83a0347a48 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:04:43.736491 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed1444-f2ad-4a4e-910e-df83a0347a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:04:43.736773 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afed1444-f2ad-4a4e-910e-df83a0347a48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:04:43.737491 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afed1444-f2ad-4a4e-910e-df83a0347a48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:04:43.739412 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afed1444-f2ad-4a4e-910e-df83a0347a48 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:04:43.851699 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1436/2857] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 20:04:43 2022] POST /identity/v3/projects => generated 391 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 20:04:43.856374 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddc79e-d8aa-49d9-b994-e87767b8ca9b admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:04:43.866433 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddc79e-d8aa-49d9-b994-e87767b8ca9b admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:04:43.922508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb2172-a137-454d-b5d1-c6991390e766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:04:43.923173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb2172-a137-454d-b5d1-c6991390e766 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:04:43.923457 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb2172-a137-454d-b5d1-c6991390e766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:04:43.923732 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb2172-a137-454d-b5d1-c6991390e766 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:04:43.924413 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb2172-a137-454d-b5d1-c6991390e766 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:04:43.926354 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb2172-a137-454d-b5d1-c6991390e766 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:04:43.933638 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:04:43.989419 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1422/2858] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 20:04:43 2022] POST /identity/v3/users => generated 296 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 20:04:43.993581 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed1444-f2ad-4a4e-910e-df83a0347a48 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:04:44.004459 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afed1444-f2ad-4a4e-910e-df83a0347a48 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:04:44.009037 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ec96d-f595-47fb-a913-ba87fb385f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:04:44.009687 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ec96d-f595-47fb-a913-ba87fb385f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:04:44.010060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ec96d-f595-47fb-a913-ba87fb385f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:04:44.010393 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ec96d-f595-47fb-a913-ba87fb385f57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:04:44.011202 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ec96d-f595-47fb-a913-ba87fb385f57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:04:44.012809 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ec96d-f595-47fb-a913-ba87fb385f57 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:04:44.023123 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1437/2859] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 20:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 20:04:44.028170 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb2172-a137-454d-b5d1-c6991390e766 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:04:44.038382 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb2172-a137-454d-b5d1-c6991390e766 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:04:44.043721 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef260a2-b0c9-4e14-932b-df25a77560dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:04:44.044375 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef260a2-b0c9-4e14-932b-df25a77560dd admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:04:44.044664 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef260a2-b0c9-4e14-932b-df25a77560dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:04:44.044945 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef260a2-b0c9-4e14-932b-df25a77560dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:04:44.045702 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef260a2-b0c9-4e14-932b-df25a77560dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:04:44.047287 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef260a2-b0c9-4e14-932b-df25a77560dd admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:04:44.057402 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1423/2860] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 20:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 20:04:44.061401 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ec96d-f595-47fb-a913-ba87fb385f57 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:04:44.071121 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ec96d-f595-47fb-a913-ba87fb385f57 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:04:44.075825 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dbae2f-7c51-425a-bf19-0a85303cb63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:04:44.076693 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbae2f-7c51-425a-bf19-0a85303cb63a admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:04:44.077001 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbae2f-7c51-425a-bf19-0a85303cb63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:04:44.077282 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbae2f-7c51-425a-bf19-0a85303cb63a admin admin] PATH_INFO: `/v3/projects/d249eff248464e19adec789d990e7fa4/users/cffc0d00082744ad8dbe04a668d4e8e1/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:04:44.091907 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dbae2f-7c51-425a-bf19-0a85303cb63a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d249eff248464e19adec789d990e7fa4, user_id=cffc0d00082744ad8dbe04a668d4e8e1, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:04:44.094597 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dbae2f-7c51-425a-bf19-0a85303cb63a admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:04:44.178075 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1438/2861] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 20:04:44 2022] PUT /identity/v3/projects/d249eff248464e19adec789d990e7fa4/users/cffc0d00082744ad8dbe04a668d4e8e1/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:04:44.188814 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4bd67-6e10-4490-badb-6904bac129ea None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:04:44.189161 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4bd67-6e10-4490-badb-6904bac129ea None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:04:44.189454 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4bd67-6e10-4490-badb-6904bac129ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:04:44.216345 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-ffd4bd67-6e10-4490-badb-6904bac129ea None None] MFA Rules not processed for user `cffc0d00082744ad8dbe04a668d4e8e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:04:44.217598 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-ffd4bd67-6e10-4490-badb-6904bac129ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:04:44.280984 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1424/2862] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:04:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 20:04:44.291664 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dbae2f-7c51-425a-bf19-0a85303cb63a admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:04:44.294378 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-73dbae2f-7c51-425a-bf19-0a85303cb63a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:04:44.309179 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:04:44.351595 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dbae2f-7c51-425a-bf19-0a85303cb63a admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:04:44.356032 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c44b40-7319-46f1-a4d0-b88fa8ca2f3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:04:44.356677 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c44b40-7319-46f1-a4d0-b88fa8ca2f3e service cinder] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:04:44.356974 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c44b40-7319-46f1-a4d0-b88fa8ca2f3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:04:44.357311 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c44b40-7319-46f1-a4d0-b88fa8ca2f3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:04:44.365715 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c44b40-7319-46f1-a4d0-b88fa8ca2f3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:04:44.367632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c44b40-7319-46f1-a4d0-b88fa8ca2f3e service cinder] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:04:44.376738 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1439/2863] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:04:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:04:54.278062 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:04:54.400838 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=20 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:05:02.975231 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4bd67-6e10-4490-badb-6904bac129ea None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:05:03.038867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4bd67-6e10-4490-badb-6904bac129ea None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:05:03.043958 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78107bf7-9317-4595-b147-6afc71e5610c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:05:03.044858 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78107bf7-9317-4595-b147-6afc71e5610c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:03.045169 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78107bf7-9317-4595-b147-6afc71e5610c admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:03.045446 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78107bf7-9317-4595-b147-6afc71e5610c admin admin] PATH_INFO: `/v3/users/cffc0d00082744ad8dbe04a668d4e8e1` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:03.046637 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78107bf7-9317-4595-b147-6afc71e5610c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cffc0d00082744ad8dbe04a668d4e8e1)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:05:03.048546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78107bf7-9317-4595-b147-6afc71e5610c admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:05:03.236784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-78107bf7-9317-4595-b147-6afc71e5610c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cffc0d00082744ad8dbe04a668d4e8e1'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:05:03.249386 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-78107bf7-9317-4595-b147-6afc71e5610c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cffc0d00082744ad8dbe04a668d4e8e1'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:05:03.314712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-78107bf7-9317-4595-b147-6afc71e5610c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cffc0d00082744ad8dbe04a668d4e8e1'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:05:03.315332 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-78107bf7-9317-4595-b147-6afc71e5610c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cffc0d00082744ad8dbe04a668d4e8e1'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:05:03.315943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-78107bf7-9317-4595-b147-6afc71e5610c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cffc0d00082744ad8dbe04a668d4e8e1'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:05:03.329322 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1425/2864] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 20:05:02 2022] DELETE /identity/v3/users/cffc0d00082744ad8dbe04a668d4e8e1 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:05:03.333832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a1e0d-154d-4db8-a022-ddf04209c13d None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:03.334351 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a1e0d-154d-4db8-a022-ddf04209c13d None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:03.334554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a1e0d-154d-4db8-a022-ddf04209c13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:03.354848 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-f21a1e0d-154d-4db8-a022-ddf04209c13d None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:05:03.356155 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-f21a1e0d-154d-4db8-a022-ddf04209c13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:05:03.415144 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1440/2865] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 20:05:03.425194 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78107bf7-9317-4595-b147-6afc71e5610c admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:05:03.426640 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-78107bf7-9317-4595-b147-6afc71e5610c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:05:03.482503 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78107bf7-9317-4595-b147-6afc71e5610c admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:05:03.486094 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2066a32-6100-4706-b712-ca5dc936a562 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:05:03.486734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2066a32-6100-4706-b712-ca5dc936a562 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:03.487072 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2066a32-6100-4706-b712-ca5dc936a562 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:03.487297 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2066a32-6100-4706-b712-ca5dc936a562 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:03.495966 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2066a32-6100-4706-b712-ca5dc936a562 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:05:03.497476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2066a32-6100-4706-b712-ca5dc936a562 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:05:03.506756 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1426/2866] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:05:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:05:03.884487 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a1e0d-154d-4db8-a022-ddf04209c13d None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:05:03.885963 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-f21a1e0d-154d-4db8-a022-ddf04209c13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:05:03.898941 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a1e0d-154d-4db8-a022-ddf04209c13d None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:05:03.904090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29841d0-8670-415f-9360-3853a36b6b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:05:03.905034 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29841d0-8670-415f-9360-3853a36b6b6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:03.905370 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29841d0-8670-415f-9360-3853a36b6b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:03.905604 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29841d0-8670-415f-9360-3853a36b6b6b admin admin] PATH_INFO: `/v3/projects/d249eff248464e19adec789d990e7fa4` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:03.906806 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29841d0-8670-415f-9360-3853a36b6b6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d249eff248464e19adec789d990e7fa4)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:05:03.908716 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29841d0-8670-415f-9360-3853a36b6b6b admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:05:04.015061 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f29841d0-8670-415f-9360-3853a36b6b6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd249eff248464e19adec789d990e7fa4', 'request_id': 'req-f29841d0-8670-415f-9360-3853a36b6b6b'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:05:04.020284 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f29841d0-8670-415f-9360-3853a36b6b6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd249eff248464e19adec789d990e7fa4', 'request_id': 'req-f29841d0-8670-415f-9360-3853a36b6b6b'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:05:04.026215 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f29841d0-8670-415f-9360-3853a36b6b6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd249eff248464e19adec789d990e7fa4', 'request_id': 'req-f29841d0-8670-415f-9360-3853a36b6b6b'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:05:04.083016 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f29841d0-8670-415f-9360-3853a36b6b6b admin admin] The token cache is being invalidate because project d249eff248464e19adec789d990e7fa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80635) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 20:05:04.083299 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f29841d0-8670-415f-9360-3853a36b6b6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:05:04.083926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-f29841d0-8670-415f-9360-3853a36b6b6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:05:04.085543 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1441/2867] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 20:05:03 2022] DELETE /identity/v3/projects/d249eff248464e19adec789d990e7fa4 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:05:04.090779 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a74524-c15e-4aa4-8e25-19e35acc97aa None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:04.094036 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a74524-c15e-4aa4-8e25-19e35acc97aa None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:04.094036 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a74524-c15e-4aa4-8e25-19e35acc97aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:04.109403 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-53a74524-c15e-4aa4-8e25-19e35acc97aa None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:05:04.110596 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-53a74524-c15e-4aa4-8e25-19e35acc97aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:05:04.127643 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:05:04.170250 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1427/2868] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:05:04.178032 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd3b8c-cab6-4e2a-bbb8-5b03cef32b4d None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:04.178328 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd3b8c-cab6-4e2a-bbb8-5b03cef32b4d None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:04.178616 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd3b8c-cab6-4e2a-bbb8-5b03cef32b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:04.197090 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-7ccd3b8c-cab6-4e2a-bbb8-5b03cef32b4d None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:05:04.198286 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-7ccd3b8c-cab6-4e2a-bbb8-5b03cef32b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:05:04.206287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1442/2869] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:05:04.210423 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a74524-c15e-4aa4-8e25-19e35acc97aa None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:05:04.220559 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a74524-c15e-4aa4-8e25-19e35acc97aa None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:05:04.225392 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec0668-3c1b-4a6e-9630-0d7156bace70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:05:04.226173 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec0668-3c1b-4a6e-9630-0d7156bace70 admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:04.226831 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec0668-3c1b-4a6e-9630-0d7156bace70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:04.227114 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec0668-3c1b-4a6e-9630-0d7156bace70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:04.227850 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec0668-3c1b-4a6e-9630-0d7156bace70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80634) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 20:05:04.228161 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec0668-3c1b-4a6e-9630-0d7156bace70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:05:04.229808 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec0668-3c1b-4a6e-9630-0d7156bace70 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:05:04.241994 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1428/2870] 10.222.0.13 () {58 vars in 1152 bytes} [Fri May 27 20:05:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 20:05:04.246032 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd3b8c-cab6-4e2a-bbb8-5b03cef32b4d None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:05:04.255274 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd3b8c-cab6-4e2a-bbb8-5b03cef32b4d None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:05:04.259835 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5afe0-d1a2-4619-9558-1d9e9cf378f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:05:04.260513 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5afe0-d1a2-4619-9558-1d9e9cf378f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:04.260798 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5afe0-d1a2-4619-9558-1d9e9cf378f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:04.261112 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea5afe0-d1a2-4619-9558-1d9e9cf378f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:04.261804 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5afe0-d1a2-4619-9558-1d9e9cf378f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:05:04.264296 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea5afe0-d1a2-4619-9558-1d9e9cf378f4 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:05:04.371368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1443/2871] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 20:05:04 2022] POST /identity/v3/projects => generated 375 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 20:05:04.376509 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec0668-3c1b-4a6e-9630-0d7156bace70 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:05:04.386573 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec0668-3c1b-4a6e-9630-0d7156bace70 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:05:04.444557 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6ca2c-b999-4d7f-be8f-2f26c07d6a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:05:04.445270 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6ca2c-b999-4d7f-be8f-2f26c07d6a07 admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:04.445557 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6ca2c-b999-4d7f-be8f-2f26c07d6a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:04.445839 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6ca2c-b999-4d7f-be8f-2f26c07d6a07 admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:04.446537 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6ca2c-b999-4d7f-be8f-2f26c07d6a07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:05:04.448458 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6ca2c-b999-4d7f-be8f-2f26c07d6a07 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:05:04.523075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1429/2872] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 20:05:04 2022] POST /identity/v3/users => generated 288 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 20:05:04.527415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5afe0-d1a2-4619-9558-1d9e9cf378f4 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:05:04.537914 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea5afe0-d1a2-4619-9558-1d9e9cf378f4 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:05:04.542586 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726690c-a5fa-4f28-866c-c36677ca7f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:05:04.550339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726690c-a5fa-4f28-866c-c36677ca7f5e admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:04.550339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726690c-a5fa-4f28-866c-c36677ca7f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:04.550339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726690c-a5fa-4f28-866c-c36677ca7f5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:04.550339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1726690c-a5fa-4f28-866c-c36677ca7f5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:05:04.550339 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1726690c-a5fa-4f28-866c-c36677ca7f5e admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:05:04.556312 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1444/2873] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 20:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 20:05:04.560784 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6ca2c-b999-4d7f-be8f-2f26c07d6a07 admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:05:04.570745 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6ca2c-b999-4d7f-be8f-2f26c07d6a07 admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:05:04.576029 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1605a4-02b2-4655-b847-ad38dc51870a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:05:04.576687 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1605a4-02b2-4655-b847-ad38dc51870a admin admin] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:04.576998 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1605a4-02b2-4655-b847-ad38dc51870a admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:04.577283 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1605a4-02b2-4655-b847-ad38dc51870a admin admin] PATH_INFO: `/v3/roles` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:04.578035 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1605a4-02b2-4655-b847-ad38dc51870a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:05:04.579669 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1605a4-02b2-4655-b847-ad38dc51870a admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:05:04.589469 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1430/2874] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 20:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 20:05:04.593688 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726690c-a5fa-4f28-866c-c36677ca7f5e admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:05:04.603518 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726690c-a5fa-4f28-866c-c36677ca7f5e admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:05:04.608233 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e004d-bd5a-4c51-918d-3dea3903f06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:05:04.609142 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e004d-bd5a-4c51-918d-3dea3903f06a admin admin] REQUEST_METHOD: `PUT` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:04.609427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e004d-bd5a-4c51-918d-3dea3903f06a admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:04.609701 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e004d-bd5a-4c51-918d-3dea3903f06a admin admin] PATH_INFO: `/v3/projects/369b202dfd5943459ccf6864e00cf1f9/users/9f022bc3e4e840519bdf4e1afd8d1aa5/roles/cb43a575e47147c799eec2f362f4fe0e` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:04.624474 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8e004d-bd5a-4c51-918d-3dea3903f06a admin admin] RBAC: Authorizing `identity:create_grant(project_id=369b202dfd5943459ccf6864e00cf1f9, user_id=9f022bc3e4e840519bdf4e1afd8d1aa5, role_id=cb43a575e47147c799eec2f362f4fe0e)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:05:04.627161 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8e004d-bd5a-4c51-918d-3dea3903f06a admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:05:04.689062 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1445/2875] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 20:05:04 2022] PUT /identity/v3/projects/369b202dfd5943459ccf6864e00cf1f9/users/9f022bc3e4e840519bdf4e1afd8d1aa5/roles/cb43a575e47147c799eec2f362f4fe0e => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:05:04.694093 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef7d63-42e5-4ab1-ae66-55fc02c8157a None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:04.694383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef7d63-42e5-4ab1-ae66-55fc02c8157a None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:04.694666 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef7d63-42e5-4ab1-ae66-55fc02c8157a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:04.712377 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-81ef7d63-42e5-4ab1-ae66-55fc02c8157a None None] MFA Rules not processed for user `736ecd08572d41d59ba0fe2db80e313a`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:05:04.713588 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-81ef7d63-42e5-4ab1-ae66-55fc02c8157a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:05:04.771040 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1431/2876] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:05:04.780802 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e004d-bd5a-4c51-918d-3dea3903f06a admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:05:04.782591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-6a8e004d-bd5a-4c51-918d-3dea3903f06a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:05:04.804368 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:05:04.839352 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e004d-bd5a-4c51-918d-3dea3903f06a admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:05:04.843389 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a650f-fec2-4ec3-a1fa-c41441b58f45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:05:04.844028 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a650f-fec2-4ec3-a1fa-c41441b58f45 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:04.844313 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a650f-fec2-4ec3-a1fa-c41441b58f45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:04.844590 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a650f-fec2-4ec3-a1fa-c41441b58f45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:04.853453 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a650f-fec2-4ec3-a1fa-c41441b58f45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:05:04.854921 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a650f-fec2-4ec3-a1fa-c41441b58f45 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:05:04.865130 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1446/2877] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:05:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:05:09.959467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb765d6-698b-4e05-bf88-79bce8dca1f6 None None] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:09.959972 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb765d6-698b-4e05-bf88-79bce8dca1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:09.960236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb765d6-698b-4e05-bf88-79bce8dca1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:09.992236 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.auth.core [None req-5fb765d6-698b-4e05-bf88-79bce8dca1f6 None None] MFA Rules not processed for user `9f022bc3e4e840519bdf4e1afd8d1aa5`. Rule list: `[]` (Enabled: `True`). {{(pid=80634) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:05:09.993502 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-5fb765d6-698b-4e05-bf88-79bce8dca1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:05:10.012813 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:05:10.063844 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1432/2878] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:05:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:05:10.068554 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a650f-fec2-4ec3-a1fa-c41441b58f45 service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:05:10.079814 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a650f-fec2-4ec3-a1fa-c41441b58f45 service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:05:10.084840 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9c52d-048f-4545-aa4c-86423647ff7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:05:10.085546 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9c52d-048f-4545-aa4c-86423647ff7b admin admin] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:10.085832 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9c52d-048f-4545-aa4c-86423647ff7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:10.086136 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9c52d-048f-4545-aa4c-86423647ff7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:10.086839 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9c52d-048f-4545-aa4c-86423647ff7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:05:10.088766 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9c52d-048f-4545-aa4c-86423647ff7b admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:05:10.175528 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1447/2879] 10.222.0.13 () {60 vars in 1152 bytes} [Fri May 27 20:05:10 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 20:05:10.179564 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb765d6-698b-4e05-bf88-79bce8dca1f6 None None] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:05:10.191943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb765d6-698b-4e05-bf88-79bce8dca1f6 None None] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:05:10.270183 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760d40a-0ff2-4b98-a5f1-e3bd8e65aa2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:05:10.271024 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760d40a-0ff2-4b98-a5f1-e3bd8e65aa2f admin admin] REQUEST_METHOD: `POST` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:10.271313 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760d40a-0ff2-4b98-a5f1-e3bd8e65aa2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:10.271594 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760d40a-0ff2-4b98-a5f1-e3bd8e65aa2f admin admin] PATH_INFO: `/v3/users` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:10.272299 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760d40a-0ff2-4b98-a5f1-e3bd8e65aa2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:05:10.274246 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760d40a-0ff2-4b98-a5f1-e3bd8e65aa2f admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:05:10.384893 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1433/2880] 10.222.0.13 () {60 vars in 1143 bytes} [Fri May 27 20:05:10 2022] POST /identity/v3/users => generated 295 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 20:05:10.388900 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9c52d-048f-4545-aa4c-86423647ff7b admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:05:10.400007 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9c52d-048f-4545-aa4c-86423647ff7b admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:05:10.405165 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f544ed6-ff8f-4069-8438-3d15a15e413c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:05:10.405877 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f544ed6-ff8f-4069-8438-3d15a15e413c admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:10.406166 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f544ed6-ff8f-4069-8438-3d15a15e413c admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:10.406448 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f544ed6-ff8f-4069-8438-3d15a15e413c admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:10.407189 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f544ed6-ff8f-4069-8438-3d15a15e413c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:05:10.408834 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f544ed6-ff8f-4069-8438-3d15a15e413c admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:05:10.421612 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1448/2881] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 20:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 20:05:10.425813 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760d40a-0ff2-4b98-a5f1-e3bd8e65aa2f admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:05:10.436371 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760d40a-0ff2-4b98-a5f1-e3bd8e65aa2f admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:05:10.441143 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c1682-e754-4cfc-b30c-5171fdd18c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:05:10.442071 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c1682-e754-4cfc-b30c-5171fdd18c9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:10.442340 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c1682-e754-4cfc-b30c-5171fdd18c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:10.442612 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c1682-e754-4cfc-b30c-5171fdd18c9a admin admin] PATH_INFO: `/v3/projects/124c9997a7874871900e2a23911c7e55/users/ece46ec9959c407888398c0b84476d88/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:10.459168 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90c1682-e754-4cfc-b30c-5171fdd18c9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=124c9997a7874871900e2a23911c7e55, user_id=ece46ec9959c407888398c0b84476d88, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:05:10.462311 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90c1682-e754-4cfc-b30c-5171fdd18c9a admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:05:10.556056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1434/2882] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 20:05:10 2022] PUT /identity/v3/projects/124c9997a7874871900e2a23911c7e55/users/ece46ec9959c407888398c0b84476d88/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:05:10.557451 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f544ed6-ff8f-4069-8438-3d15a15e413c admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:05:10.570569 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f544ed6-ff8f-4069-8438-3d15a15e413c admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:05:10.635927 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592419e-1894-40d8-b55b-0fcbe5ec9845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:05:10.636617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592419e-1894-40d8-b55b-0fcbe5ec9845 admin admin] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:10.636992 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592419e-1894-40d8-b55b-0fcbe5ec9845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:10.637343 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592419e-1894-40d8-b55b-0fcbe5ec9845 admin admin] PATH_INFO: `/v3/roles` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:10.638908 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592419e-1894-40d8-b55b-0fcbe5ec9845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:05:10.640680 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592419e-1894-40d8-b55b-0fcbe5ec9845 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:05:10.652337 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1449/2883] 10.222.0.13 () {58 vars in 1121 bytes} [Fri May 27 20:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 20:05:10.657341 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c1682-e754-4cfc-b30c-5171fdd18c9a admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:05:10.668117 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c1682-e754-4cfc-b30c-5171fdd18c9a admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:05:10.674263 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044aeaf-a3a2-49f0-940d-24bab3445dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:05:10.675304 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044aeaf-a3a2-49f0-940d-24bab3445dbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:10.675655 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044aeaf-a3a2-49f0-940d-24bab3445dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:10.675999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6044aeaf-a3a2-49f0-940d-24bab3445dbf admin admin] PATH_INFO: `/v3/projects/124c9997a7874871900e2a23911c7e55/users/ece46ec9959c407888398c0b84476d88/roles/6505bbb129404436966ee59aa7a01d8e` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:10.678182 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6044aeaf-a3a2-49f0-940d-24bab3445dbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=124c9997a7874871900e2a23911c7e55, user_id=ece46ec9959c407888398c0b84476d88, role_id=6505bbb129404436966ee59aa7a01d8e)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:05:10.681082 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6044aeaf-a3a2-49f0-940d-24bab3445dbf admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:05:10.697745 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1435/2884] 10.222.0.13 () {60 vars in 1482 bytes} [Fri May 27 20:05:10 2022] PUT /identity/v3/projects/124c9997a7874871900e2a23911c7e55/users/ece46ec9959c407888398c0b84476d88/roles/6505bbb129404436966ee59aa7a01d8e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:05:16.004415 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdecb657-1fea-4954-9111-e98e34ad0357 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:16.005154 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdecb657-1fea-4954-9111-e98e34ad0357 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:16.005454 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdecb657-1fea-4954-9111-e98e34ad0357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:16.035467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-fdecb657-1fea-4954-9111-e98e34ad0357 None None] MFA Rules not processed for user `ece46ec9959c407888398c0b84476d88`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:05:16.036755 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-fdecb657-1fea-4954-9111-e98e34ad0357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:05:16.124129 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1450/2885] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:05:16 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:05:16.139300 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044aeaf-a3a2-49f0-940d-24bab3445dbf admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:05:16.141258 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-6044aeaf-a3a2-49f0-940d-24bab3445dbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:05:16.219060 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:05:16.246323 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6044aeaf-a3a2-49f0-940d-24bab3445dbf admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:05:16.253278 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1089c-0521-4291-83a8-3b87a62c37a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:05:16.253999 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1089c-0521-4291-83a8-3b87a62c37a3 service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:16.254239 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1089c-0521-4291-83a8-3b87a62c37a3 service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:16.254516 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1089c-0521-4291-83a8-3b87a62c37a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:16.313771 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1089c-0521-4291-83a8-3b87a62c37a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:05:16.315306 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1089c-0521-4291-83a8-3b87a62c37a3 service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:05:16.325329 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1436/2886] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:05:16 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:05:26.119647 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:05:26.387735 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=33 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:05:34.462115 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790c39a-1b5f-4545-af36-cfae03e206c9 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:34.462652 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790c39a-1b5f-4545-af36-cfae03e206c9 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:34.462652 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790c39a-1b5f-4545-af36-cfae03e206c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:34.480209 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-3790c39a-1b5f-4545-af36-cfae03e206c9 None None] MFA Rules not processed for user `9f022bc3e4e840519bdf4e1afd8d1aa5`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:05:34.481579 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-3790c39a-1b5f-4545-af36-cfae03e206c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:05:34.488427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1451/2887] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:05:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:05:34.526646 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1089c-0521-4291-83a8-3b87a62c37a3 service nova] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:05:34.528190 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-72f1089c-0521-4291-83a8-3b87a62c37a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:05:34.587951 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1089c-0521-4291-83a8-3b87a62c37a3 service nova] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:05:34.592736 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056ef47-dd0d-4e01-bbbc-c8998c8aeeec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7b7e90ca75d643cbb0e6d6791fc30b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:05:34.593667 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056ef47-dd0d-4e01-bbbc-c8998c8aeeec service cinder] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:34.593956 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056ef47-dd0d-4e01-bbbc-c8998c8aeeec service cinder] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:34.594239 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056ef47-dd0d-4e01-bbbc-c8998c8aeeec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:34.596056 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-7056ef47-dd0d-4e01-bbbc-c8998c8aeeec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:05:34.612292 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056ef47-dd0d-4e01-bbbc-c8998c8aeeec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:05:34.613712 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056ef47-dd0d-4e01-bbbc-c8998c8aeeec service cinder] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:05:34.625557 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1437/2888] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:05:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:05:34.904538 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790c39a-1b5f-4545-af36-cfae03e206c9 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:05:34.958935 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790c39a-1b5f-4545-af36-cfae03e206c9 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:05:34.962537 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17466e-b89e-4949-a59f-163dbbddbe51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:05:34.963182 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17466e-b89e-4949-a59f-163dbbddbe51 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:34.963467 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17466e-b89e-4949-a59f-163dbbddbe51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:34.963745 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f17466e-b89e-4949-a59f-163dbbddbe51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:34.965530 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-2f17466e-b89e-4949-a59f-163dbbddbe51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:05:34.975867 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17466e-b89e-4949-a59f-163dbbddbe51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:05:34.977360 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f17466e-b89e-4949-a59f-163dbbddbe51 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:05:34.985379 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1452/2889] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:05:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:05:35.682084 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056ef47-dd0d-4e01-bbbc-c8998c8aeeec service cinder] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:05:35.683537 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-7056ef47-dd0d-4e01-bbbc-c8998c8aeeec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:05:35.695804 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056ef47-dd0d-4e01-bbbc-c8998c8aeeec service cinder] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:05:35.699853 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd83c39-fb73-4c71-ac2e-9c4b8121bd85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:05:35.700503 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd83c39-fb73-4c71-ac2e-9c4b8121bd85 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:35.700787 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd83c39-fb73-4c71-ac2e-9c4b8121bd85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:35.701118 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd83c39-fb73-4c71-ac2e-9c4b8121bd85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:35.702926 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-ecd83c39-fb73-4c71-ac2e-9c4b8121bd85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:05:35.717801 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd83c39-fb73-4c71-ac2e-9c4b8121bd85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:05:35.719586 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd83c39-fb73-4c71-ac2e-9c4b8121bd85 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:05:35.732000 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1438/2890] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:05:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 20:05:44.981828 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=29 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:05:45.726856 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=32 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:05:51.377029 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17466e-b89e-4949-a59f-163dbbddbe51 service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:05:51.388650 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f17466e-b89e-4949-a59f-163dbbddbe51 service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:05:51.392515 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312f443-da13-425a-9dea-2f2602e75352 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:05:51.393192 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312f443-da13-425a-9dea-2f2602e75352 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:51.393477 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312f443-da13-425a-9dea-2f2602e75352 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:51.393764 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0312f443-da13-425a-9dea-2f2602e75352 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:51.395562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-0312f443-da13-425a-9dea-2f2602e75352 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:05:51.405947 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312f443-da13-425a-9dea-2f2602e75352 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:05:51.407367 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0312f443-da13-425a-9dea-2f2602e75352 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:05:51.415488 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1453/2891] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:05:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:05:53.474543 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd83c39-fb73-4c71-ac2e-9c4b8121bd85 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:05:53.483769 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd83c39-fb73-4c71-ac2e-9c4b8121bd85 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:05:53.487440 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e94d7f5-5808-4268-ac19-65eb470461ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:05:53.488078 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94d7f5-5808-4268-ac19-65eb470461ab service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:53.488363 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94d7f5-5808-4268-ac19-65eb470461ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:53.488645 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e94d7f5-5808-4268-ac19-65eb470461ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:53.499971 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e94d7f5-5808-4268-ac19-65eb470461ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:05:53.501493 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e94d7f5-5808-4268-ac19-65eb470461ab service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:05:53.511478 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1439/2892] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:05:53 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:05:55.451896 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312f443-da13-425a-9dea-2f2602e75352 service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:05:55.453479 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-0312f443-da13-425a-9dea-2f2602e75352 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:05:55.470511 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0312f443-da13-425a-9dea-2f2602e75352 service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:05:55.475968 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b76313-27b6-4bff-9e09-195eef4c2fe3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:05:55.476632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b76313-27b6-4bff-9e09-195eef4c2fe3 service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:05:55.476917 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b76313-27b6-4bff-9e09-195eef4c2fe3 service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:05:55.477271 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b76313-27b6-4bff-9e09-195eef4c2fe3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:05:55.479291 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-89b76313-27b6-4bff-9e09-195eef4c2fe3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:05:55.492127 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b76313-27b6-4bff-9e09-195eef4c2fe3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:05:55.494508 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b76313-27b6-4bff-9e09-195eef4c2fe3 service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:05:55.506434 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1454/2893] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:05:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:06:03.506825 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=6 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:06:05.501217 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=12 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:07:52.535201 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e94d7f5-5808-4268-ac19-65eb470461ab service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:07:52.548622 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e94d7f5-5808-4268-ac19-65eb470461ab service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:07:52.555761 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec39c93-a7de-48d8-8e63-d23eb87cb8cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:07:52.556416 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec39c93-a7de-48d8-8e63-d23eb87cb8cf service nova] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:07:52.556692 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec39c93-a7de-48d8-8e63-d23eb87cb8cf service nova] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:07:52.556990 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec39c93-a7de-48d8-8e63-d23eb87cb8cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:07:52.558943 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-bec39c93-a7de-48d8-8e63-d23eb87cb8cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:07:52.573562 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec39c93-a7de-48d8-8e63-d23eb87cb8cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:07:52.575176 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec39c93-a7de-48d8-8e63-d23eb87cb8cf service nova] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:07:52.587401 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1440/2894] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:07:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:07:54.783991 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7273fc-a3ef-42f1-b8d7-2b9505392baf None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:07:54.784472 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7273fc-a3ef-42f1-b8d7-2b9505392baf None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:07:54.784591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7273fc-a3ef-42f1-b8d7-2b9505392baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:07:54.802202 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-ab7273fc-a3ef-42f1-b8d7-2b9505392baf None None] MFA Rules not processed for user `9f022bc3e4e840519bdf4e1afd8d1aa5`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:07:54.804060 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-ab7273fc-a3ef-42f1-b8d7-2b9505392baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:07:54.811160 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1455/2895] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:07:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:07:57.823081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec39c93-a7de-48d8-8e63-d23eb87cb8cf service nova] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:07:57.824688 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-bec39c93-a7de-48d8-8e63-d23eb87cb8cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:07:57.895545 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec39c93-a7de-48d8-8e63-d23eb87cb8cf service nova] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:07:57.901166 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c7fd8-92c1-4947-848b-684fc6438996 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2ed13a0c84f128b795002892efc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:07:57.901814 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c7fd8-92c1-4947-848b-684fc6438996 service placement] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:07:57.902104 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c7fd8-92c1-4947-848b-684fc6438996 service placement] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:07:57.902383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c7fd8-92c1-4947-848b-684fc6438996 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:07:57.904164 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-076c7fd8-92c1-4947-848b-684fc6438996 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:07:57.916099 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c7fd8-92c1-4947-848b-684fc6438996 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:07:57.917803 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c7fd8-92c1-4947-848b-684fc6438996 service placement] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:07:57.929962 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1441/2896] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:07:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:08:04.797174 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:08:07.923454 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=38 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:08:15.970917 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7273fc-a3ef-42f1-b8d7-2b9505392baf None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:08:15.984272 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7273fc-a3ef-42f1-b8d7-2b9505392baf None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:08:15.989826 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a4fb2-d49d-4ea4-8a15-10501db483c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:08:15.990470 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a4fb2-d49d-4ea4-8a15-10501db483c1 service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:08:15.990752 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a4fb2-d49d-4ea4-8a15-10501db483c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:08:15.991026 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a4fb2-d49d-4ea4-8a15-10501db483c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:08:15.993257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-0d8a4fb2-d49d-4ea4-8a15-10501db483c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:08:16.005180 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8a4fb2-d49d-4ea4-8a15-10501db483c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:08:16.006752 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8a4fb2-d49d-4ea4-8a15-10501db483c1 service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:08:16.017624 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1456/2897] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:08:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:08:26.013208 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=6 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:09:35.700754 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c7fd8-92c1-4947-848b-684fc6438996 service placement] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:09:35.702183 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-076c7fd8-92c1-4947-848b-684fc6438996 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:09:35.715472 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c7fd8-92c1-4947-848b-684fc6438996 service placement] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:09:35.720408 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d84fb-b79d-439e-9473-5d0b95596787 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2ed13a0c84f128b795002892efc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:09:35.721073 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d84fb-b79d-439e-9473-5d0b95596787 service placement] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:09:35.721361 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d84fb-b79d-439e-9473-5d0b95596787 service placement] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:09:35.721639 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44d84fb-b79d-439e-9473-5d0b95596787 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:09:35.724081 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-c44d84fb-b79d-439e-9473-5d0b95596787 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:09:35.735515 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44d84fb-b79d-439e-9473-5d0b95596787 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:09:35.737118 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44d84fb-b79d-439e-9473-5d0b95596787 service placement] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:09:35.746881 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1442/2898] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:09:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:09:45.742623 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=6 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:09:55.230928 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483c4da-52b3-4c09-b29b-986220b0f5c5 None None] REQUEST_METHOD: `POST` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:09:55.231458 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483c4da-52b3-4c09-b29b-986220b0f5c5 None None] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:09:55.231574 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b483c4da-52b3-4c09-b29b-986220b0f5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:09:55.249427 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.auth.core [None req-b483c4da-52b3-4c09-b29b-986220b0f5c5 None None] MFA Rules not processed for user `9f022bc3e4e840519bdf4e1afd8d1aa5`. Rule list: `[]` (Enabled: `True`). {{(pid=80635) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 20:09:55.250617 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-b483c4da-52b3-4c09-b29b-986220b0f5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:09:55.258142 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1457/2899] 10.222.0.13 () {58 vars in 957 bytes} [Fri May 27 20:09:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 20:09:55.573731 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d84fb-b79d-439e-9473-5d0b95596787 service placement] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:09:55.583225 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44d84fb-b79d-439e-9473-5d0b95596787 service placement] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:09:55.586800 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e16eed3-feba-478d-9c78-43c88efed789 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:09:55.587467 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16eed3-feba-478d-9c78-43c88efed789 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:09:55.587781 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16eed3-feba-478d-9c78-43c88efed789 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:09:55.588072 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e16eed3-feba-478d-9c78-43c88efed789 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:09:55.589855 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-9e16eed3-feba-478d-9c78-43c88efed789 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:09:55.599894 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e16eed3-feba-478d-9c78-43c88efed789 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:09:55.601361 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e16eed3-feba-478d-9c78-43c88efed789 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:09:55.609444 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1443/2900] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:09:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:10:05.244812 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=3 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:10:05.605589 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=6 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:11:29.450997 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483c4da-52b3-4c09-b29b-986220b0f5c5 None None] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:11:29.460933 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b483c4da-52b3-4c09-b29b-986220b0f5c5 None None] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:11:29.464685 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c91739-87f7-4943-a42c-15af182fce6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:11:29.465374 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c91739-87f7-4943-a42c-15af182fce6d service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:11:29.465659 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c91739-87f7-4943-a42c-15af182fce6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:11:29.465940 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c91739-87f7-4943-a42c-15af182fce6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:11:29.473380 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c91739-87f7-4943-a42c-15af182fce6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:11:29.475450 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c91739-87f7-4943-a42c-15af182fce6d service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:11:29.483591 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1458/2901] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:11:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:11:39.479907 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=6 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:11:44.835296 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e16eed3-feba-478d-9c78-43c88efed789 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:11:44.844006 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e16eed3-feba-478d-9c78-43c88efed789 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:11:44.847605 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce60c8-8f66-4969-9032-582d0db3c410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:11:44.848240 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce60c8-8f66-4969-9032-582d0db3c410 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:11:44.848523 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce60c8-8f66-4969-9032-582d0db3c410 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:11:44.848808 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ce60c8-8f66-4969-9032-582d0db3c410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:11:44.857560 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ce60c8-8f66-4969-9032-582d0db3c410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:11:44.859019 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ce60c8-8f66-4969-9032-582d0db3c410 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:11:44.868714 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1444/2902] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:11:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:11:46.057287 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c91739-87f7-4943-a42c-15af182fce6d service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:11:46.072082 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c91739-87f7-4943-a42c-15af182fce6d service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:11:46.078326 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b33d9-6f42-4bbf-97cb-6ab1e1685e6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7542e1fd160e4b2c9d25accf19aad19f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:11:46.079387 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b33d9-6f42-4bbf-97cb-6ab1e1685e6a service nova] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:11:46.079387 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b33d9-6f42-4bbf-97cb-6ab1e1685e6a service nova] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:11:46.079615 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12b33d9-6f42-4bbf-97cb-6ab1e1685e6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:11:46.093405 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12b33d9-6f42-4bbf-97cb-6ab1e1685e6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:11:46.093405 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12b33d9-6f42-4bbf-97cb-6ab1e1685e6a service nova] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:11:46.104908 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1459/2903] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:11:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:11:46.473797 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce60c8-8f66-4969-9032-582d0db3c410 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:11:46.482862 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ce60c8-8f66-4969-9032-582d0db3c410 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:11:46.486783 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e344c-8ab8-44e6-8d1d-3d950b72002c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:11:46.487447 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e344c-8ab8-44e6-8d1d-3d950b72002c service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:11:46.487714 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e344c-8ab8-44e6-8d1d-3d950b72002c service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:11:46.488004 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e344c-8ab8-44e6-8d1d-3d950b72002c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:11:46.499036 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e344c-8ab8-44e6-8d1d-3d950b72002c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:11:46.500571 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e344c-8ab8-44e6-8d1d-3d950b72002c service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:11:46.513421 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1445/2904] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:11:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:11:47.692562 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b33d9-6f42-4bbf-97cb-6ab1e1685e6a service nova] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:11:47.708683 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12b33d9-6f42-4bbf-97cb-6ab1e1685e6a service nova] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:11:47.712934 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba42be2-af2f-46b1-8ac4-b748fb2133c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:11:47.713601 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba42be2-af2f-46b1-8ac4-b748fb2133c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:11:47.713886 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba42be2-af2f-46b1-8ac4-b748fb2133c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:11:47.714164 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba42be2-af2f-46b1-8ac4-b748fb2133c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:11:47.745819 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba42be2-af2f-46b1-8ac4-b748fb2133c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:11:47.747193 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba42be2-af2f-46b1-8ac4-b748fb2133c6 service neutron] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:11:47.758882 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1460/2905] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:11:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:11:56.508535 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=12 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:11:57.753277 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=24 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:11:59.212296 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e344c-8ab8-44e6-8d1d-3d950b72002c service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:11:59.231286 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e344c-8ab8-44e6-8d1d-3d950b72002c service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:11:59.234867 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab2eed-fcef-42d0-9941-39c736b6ff42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dd34f15400c4476bbb2161dbe1b8b2b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:11:59.235751 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab2eed-fcef-42d0-9941-39c736b6ff42 service neutron] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:11:59.236045 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab2eed-fcef-42d0-9941-39c736b6ff42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:11:59.236323 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab2eed-fcef-42d0-9941-39c736b6ff42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:11:59.306734 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab2eed-fcef-42d0-9941-39c736b6ff42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:11:59.308205 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab2eed-fcef-42d0-9941-39c736b6ff42 service neutron] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:11:59.326835 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1446/2906] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:11:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 20:12:02.511669 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba42be2-af2f-46b1-8ac4-b748fb2133c6 service neutron] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:12:02.530021 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba42be2-af2f-46b1-8ac4-b748fb2133c6 service neutron] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:12:02.535481 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf729f85-4bec-459d-a7f4-74f8202a30f7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2ed13a0c84f128b795002892efc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:12:02.536167 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf729f85-4bec-459d-a7f4-74f8202a30f7 service placement] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:12:02.536453 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf729f85-4bec-459d-a7f4-74f8202a30f7 service placement] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:12:02.536735 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf729f85-4bec-459d-a7f4-74f8202a30f7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:12:02.538632 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-cf729f85-4bec-459d-a7f4-74f8202a30f7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:12:02.572898 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf729f85-4bec-459d-a7f4-74f8202a30f7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:12:02.574533 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf729f85-4bec-459d-a7f4-74f8202a30f7 service placement] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:12:02.584636 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1461/2907] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:12:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:12:06.148562 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab2eed-fcef-42d0-9941-39c736b6ff42 service neutron] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:12:06.159908 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab2eed-fcef-42d0-9941-39c736b6ff42 service neutron] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:12:06.165122 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960b23b-22cf-4d28-b154-07602e72d239 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2ed13a0c84f128b795002892efc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:12:06.165764 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960b23b-22cf-4d28-b154-07602e72d239 service placement] REQUEST_METHOD: `GET` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:12:06.166075 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960b23b-22cf-4d28-b154-07602e72d239 service placement] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:12:06.166347 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960b23b-22cf-4d28-b154-07602e72d239 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:12:06.168185 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-4960b23b-22cf-4d28-b154-07602e72d239 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:12:06.179568 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960b23b-22cf-4d28-b154-07602e72d239 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:12:06.181860 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960b23b-22cf-4d28-b154-07602e72d239 service placement] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:12:06.193521 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1447/2908] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:12:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:12:06.272541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf729f85-4bec-459d-a7f4-74f8202a30f7 service placement] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:12:06.283216 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf729f85-4bec-459d-a7f4-74f8202a30f7 service placement] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:12:06.287754 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6e25d-4275-4732-a9f1-de03941c45c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:12:06.288625 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6e25d-4275-4732-a9f1-de03941c45c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:12:06.288909 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6e25d-4275-4732-a9f1-de03941c45c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:12:06.289271 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6e25d-4275-4732-a9f1-de03941c45c9 admin admin] PATH_INFO: `/v3/users/9f022bc3e4e840519bdf4e1afd8d1aa5` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:12:06.290471 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6e25d-4275-4732-a9f1-de03941c45c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f022bc3e4e840519bdf4e1afd8d1aa5)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:12:06.292402 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6e25d-4275-4732-a9f1-de03941c45c9 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:12:06.469877 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d5b6e25d-4275-4732-a9f1-de03941c45c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f022bc3e4e840519bdf4e1afd8d1aa5'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:12:06.482512 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d5b6e25d-4275-4732-a9f1-de03941c45c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f022bc3e4e840519bdf4e1afd8d1aa5'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:12:06.570043 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d5b6e25d-4275-4732-a9f1-de03941c45c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f022bc3e4e840519bdf4e1afd8d1aa5'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:12:06.571028 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d5b6e25d-4275-4732-a9f1-de03941c45c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f022bc3e4e840519bdf4e1afd8d1aa5'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:12:06.578668 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-d5b6e25d-4275-4732-a9f1-de03941c45c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f022bc3e4e840519bdf4e1afd8d1aa5'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:12:06.584577 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1462/2909] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 20:12:06 2022] DELETE /identity/v3/users/9f022bc3e4e840519bdf4e1afd8d1aa5 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:12:06.835336 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960b23b-22cf-4d28-b154-07602e72d239 service placement] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:12:06.845476 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960b23b-22cf-4d28-b154-07602e72d239 service placement] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:12:06.907119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48fd8de-03fe-484f-8180-21cbd698882b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:12:06.907119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fd8de-03fe-484f-8180-21cbd698882b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:12:06.907119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fd8de-03fe-484f-8180-21cbd698882b admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:12:06.907119 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fd8de-03fe-484f-8180-21cbd698882b admin admin] PATH_INFO: `/v3/projects/369b202dfd5943459ccf6864e00cf1f9` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:12:06.908150 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48fd8de-03fe-484f-8180-21cbd698882b admin admin] RBAC: Authorizing `identity:delete_project(project_id=369b202dfd5943459ccf6864e00cf1f9)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:12:06.910202 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48fd8de-03fe-484f-8180-21cbd698882b admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:12:07.013805 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b48fd8de-03fe-484f-8180-21cbd698882b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '369b202dfd5943459ccf6864e00cf1f9', 'request_id': 'req-b48fd8de-03fe-484f-8180-21cbd698882b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:12:07.017931 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:12:07.020386 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b48fd8de-03fe-484f-8180-21cbd698882b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '369b202dfd5943459ccf6864e00cf1f9', 'request_id': 'req-b48fd8de-03fe-484f-8180-21cbd698882b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:12:07.078824 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b48fd8de-03fe-484f-8180-21cbd698882b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '369b202dfd5943459ccf6864e00cf1f9', 'request_id': 'req-b48fd8de-03fe-484f-8180-21cbd698882b'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:12:07.104508 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b48fd8de-03fe-484f-8180-21cbd698882b admin admin] The token cache is being invalidate because project 369b202dfd5943459ccf6864e00cf1f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 20:12:07.104801 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b48fd8de-03fe-484f-8180-21cbd698882b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:12:07.105483 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-b48fd8de-03fe-484f-8180-21cbd698882b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:12:07.107070 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1448/2910] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 20:12:06 2022] DELETE /identity/v3/projects/369b202dfd5943459ccf6864e00cf1f9 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:12:07.111341 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6e25d-4275-4732-a9f1-de03941c45c9 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:12:07.112796 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-d5b6e25d-4275-4732-a9f1-de03941c45c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:12:07.126028 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6e25d-4275-4732-a9f1-de03941c45c9 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:12:07.182516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34785f52-7eae-4afc-82dd-e4ae4f6205c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:12:07.183385 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34785f52-7eae-4afc-82dd-e4ae4f6205c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:12:07.183673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34785f52-7eae-4afc-82dd-e4ae4f6205c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:12:07.183955 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34785f52-7eae-4afc-82dd-e4ae4f6205c8 admin admin] PATH_INFO: `/v3/users/ece46ec9959c407888398c0b84476d88` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:12:07.185170 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34785f52-7eae-4afc-82dd-e4ae4f6205c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ece46ec9959c407888398c0b84476d88)` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:12:07.187103 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34785f52-7eae-4afc-82dd-e4ae4f6205c8 admin admin] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:12:07.400336 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:12:07.406062 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-34785f52-7eae-4afc-82dd-e4ae4f6205c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ece46ec9959c407888398c0b84476d88'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:12:07.417679 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-34785f52-7eae-4afc-82dd-e4ae4f6205c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ece46ec9959c407888398c0b84476d88'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:12:07.512933 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-34785f52-7eae-4afc-82dd-e4ae4f6205c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ece46ec9959c407888398c0b84476d88'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:12:07.513603 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-34785f52-7eae-4afc-82dd-e4ae4f6205c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ece46ec9959c407888398c0b84476d88'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:12:07.521257 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.notifications [None req-34785f52-7eae-4afc-82dd-e4ae4f6205c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ece46ec9959c407888398c0b84476d88'} {{(pid=80635) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:12:07.528056 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1463/2911] 10.222.0.13 () {58 vars in 1223 bytes} [Fri May 27 20:12:07 2022] DELETE /identity/v3/users/ece46ec9959c407888398c0b84476d88 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:12:07.770318 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48fd8de-03fe-484f-8180-21cbd698882b admin admin] Authenticating user token {{(pid=80634) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:12:07.771877 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.fernet_utils [None req-b48fd8de-03fe-484f-8180-21cbd698882b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80634) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:12:07.785432 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48fd8de-03fe-484f-8180-21cbd698882b admin admin] Validating token access rules against request {{(pid=80634) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:12:07.844095 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27858a17-639a-47d6-92b8-0c67f259f3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '736ecd08572d41d59ba0fe2db80e313a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6c2fc958121b4494a2ef337a1cb0b9b4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80634) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:12:07.844986 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27858a17-639a-47d6-92b8-0c67f259f3a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:12:07.845274 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27858a17-639a-47d6-92b8-0c67f259f3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:12:07.845546 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27858a17-639a-47d6-92b8-0c67f259f3a7 admin admin] PATH_INFO: `/v3/projects/124c9997a7874871900e2a23911c7e55` {{(pid=80634) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:12:07.846745 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27858a17-639a-47d6-92b8-0c67f259f3a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=124c9997a7874871900e2a23911c7e55)` {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:12:07.848674 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27858a17-639a-47d6-92b8-0c67f259f3a7 admin admin] RBAC: Authorization granted {{(pid=80634) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:12:07.998762 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-27858a17-639a-47d6-92b8-0c67f259f3a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '124c9997a7874871900e2a23911c7e55', 'request_id': 'req-27858a17-639a-47d6-92b8-0c67f259f3a7'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:12:08.004151 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-27858a17-639a-47d6-92b8-0c67f259f3a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '124c9997a7874871900e2a23911c7e55', 'request_id': 'req-27858a17-639a-47d6-92b8-0c67f259f3a7'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:12:08.078412 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-27858a17-639a-47d6-92b8-0c67f259f3a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '124c9997a7874871900e2a23911c7e55', 'request_id': 'req-27858a17-639a-47d6-92b8-0c67f259f3a7'} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:12:08.090208 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-27858a17-639a-47d6-92b8-0c67f259f3a7 admin admin] The token cache is being invalidate because project 124c9997a7874871900e2a23911c7e55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80634) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 20:12:08.090524 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-27858a17-639a-47d6-92b8-0c67f259f3a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:12:08.098383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG keystone.notifications [None req-27858a17-639a-47d6-92b8-0c67f259f3a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80634) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 20:12:08.098383 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: [pid: 80634|app: 0|req: 1449/2912] 10.222.0.13 () {58 vars in 1232 bytes} [Fri May 27 20:12:07 2022] DELETE /identity/v3/projects/124c9997a7874871900e2a23911c7e55 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 20:12:17.592926 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:DELETE=5,keystone:SELECT=14,keystone:INSERT=1 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:12:18.087234 dv-cinder-839793-12-iscsi devstack@keystone.service[80634]: DEBUG dbcounter [-] [80634] Writing DB stats keystone:DELETE=9,keystone:SELECT=61,keystone:INSERT=2 {{(pid=80634) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 20:14:41.697673 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34785f52-7eae-4afc-82dd-e4ae4f6205c8 admin admin] Authenticating user token {{(pid=80635) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 20:14:41.699078 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-34785f52-7eae-4afc-82dd-e4ae4f6205c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:14:41.788727 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34785f52-7eae-4afc-82dd-e4ae4f6205c8 admin admin] Validating token access rules against request {{(pid=80635) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 20:14:41.845893 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057698c-87ee-4ded-b88c-c8cc0900dc02 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffc2ed13a0c84f128b795002892efc2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47d6b6971c3452e9799e8146856bdc5', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80635) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 20:14:41.846541 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057698c-87ee-4ded-b88c-c8cc0900dc02 service placement] REQUEST_METHOD: `GET` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 20:14:41.846827 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057698c-87ee-4ded-b88c-c8cc0900dc02 service placement] SCRIPT_NAME: `/identity` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 20:14:41.847104 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057698c-87ee-4ded-b88c-c8cc0900dc02 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80635) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 20:14:41.848885 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.fernet_utils [None req-1057698c-87ee-4ded-b88c-c8cc0900dc02 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80635) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 20:14:41.860516 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1057698c-87ee-4ded-b88c-c8cc0900dc02 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 20:14:41.862413 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1057698c-87ee-4ded-b88c-c8cc0900dc02 service placement] RBAC: Authorization granted {{(pid=80635) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 20:14:41.872099 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: [pid: 80635|app: 0|req: 1464/2913] 10.222.0.13 () {60 vars in 1365 bytes} [Fri May 27 20:14:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 20:14:51.868012 dv-cinder-839793-12-iscsi devstack@keystone.service[80635]: DEBUG dbcounter [-] [80635] Writing DB stats keystone:SELECT=67 {{(pid=80635) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}